2023-03-08 07:53:30,049 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-08 07:53:30,302 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491 2023-03-08 07:53:30,319 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-08 07:53:30,320 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/cluster_884a5b1f-9982-fdc0-2356-490fbc1a75f2, deleteOnExit=true 2023-03-08 07:53:30,320 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-08 07:53:30,321 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/test.cache.data in system properties and HBase conf 2023-03-08 07:53:30,321 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/hadoop.tmp.dir in system properties and HBase conf 2023-03-08 07:53:30,322 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/hadoop.log.dir in system properties and HBase conf 2023-03-08 07:53:30,323 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-08 07:53:30,324 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-08 07:53:30,324 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-08 07:53:30,440 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-08 07:53:30,864 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-08 07:53:30,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-08 07:53:30,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-08 07:53:30,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-08 07:53:30,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 07:53:30,869 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-08 07:53:30,869 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-08 07:53:30,869 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 07:53:30,870 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 07:53:30,870 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-08 07:53:30,870 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/nfs.dump.dir in system properties and HBase conf 2023-03-08 07:53:30,870 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/java.io.tmpdir in system properties and HBase conf 2023-03-08 07:53:30,871 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 07:53:30,871 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-08 07:53:30,871 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-08 07:53:31,441 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 07:53:31,446 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 07:53:31,754 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-08 07:53:31,960 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-08 07:53:31,980 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 07:53:32,020 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 07:53:32,054 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/java.io.tmpdir/Jetty_localhost_45703_hdfs____.tdsc1p/webapp 2023-03-08 07:53:32,198 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45703 2023-03-08 07:53:32,237 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 07:53:32,237 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 07:53:32,648 WARN [Listener at localhost/39077] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 07:53:32,796 WARN [Listener at localhost/39077] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-08 07:53:32,822 WARN [Listener at localhost/39077] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 07:53:32,829 INFO [Listener at localhost/39077] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 07:53:32,835 INFO [Listener at localhost/39077] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/java.io.tmpdir/Jetty_localhost_34383_datanode____ld82j8/webapp 2023-03-08 07:53:32,949 INFO [Listener at localhost/39077] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34383 2023-03-08 07:53:33,351 WARN [Listener at localhost/42999] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 07:53:33,922 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe0932c38b2815ff0: Processing first storage report for DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321 from datanode 45ea35ad-08cb-4f1a-a4db-6690f1026251 2023-03-08 07:53:33,926 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe0932c38b2815ff0: from storage DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321 node DatanodeRegistration(127.0.0.1:40571, datanodeUuid=45ea35ad-08cb-4f1a-a4db-6690f1026251, infoPort=34327, infoSecurePort=0, ipcPort=42999, storageInfo=lv=-57;cid=testClusterID;nsid=778962744;c=1678262011522), blocks: 0, hasStaleStorage: true, processing time: 4 msecs, invalidatedBlocks: 0 2023-03-08 07:53:33,927 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe0932c38b2815ff0: Processing first storage report for DS-2b992472-7ef0-4ef1-87f1-254a6606f3ee from datanode 45ea35ad-08cb-4f1a-a4db-6690f1026251 2023-03-08 07:53:33,927 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe0932c38b2815ff0: from storage DS-2b992472-7ef0-4ef1-87f1-254a6606f3ee node DatanodeRegistration(127.0.0.1:40571, datanodeUuid=45ea35ad-08cb-4f1a-a4db-6690f1026251, infoPort=34327, infoSecurePort=0, ipcPort=42999, storageInfo=lv=-57;cid=testClusterID;nsid=778962744;c=1678262011522), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-08 07:53:33,990 DEBUG [Listener at localhost/42999] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491 2023-03-08 07:53:34,057 INFO [Listener at localhost/42999] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/cluster_884a5b1f-9982-fdc0-2356-490fbc1a75f2/zookeeper_0, clientPort=52387, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/cluster_884a5b1f-9982-fdc0-2356-490fbc1a75f2/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/cluster_884a5b1f-9982-fdc0-2356-490fbc1a75f2/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-08 07:53:34,071 INFO [Listener at localhost/42999] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52387 2023-03-08 07:53:34,082 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:34,085 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:34,755 INFO [Listener at localhost/42999] util.FSUtils(471): Created version file at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b with version=8 2023-03-08 07:53:34,755 INFO [Listener at localhost/42999] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/hbase-staging 2023-03-08 07:53:35,067 INFO [Listener at localhost/42999] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-08 07:53:35,528 INFO [Listener at localhost/42999] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-08 07:53:35,560 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 07:53:35,561 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 07:53:35,561 INFO [Listener at localhost/42999] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 07:53:35,561 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 07:53:35,561 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 07:53:35,703 INFO [Listener at localhost/42999] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 07:53:35,782 DEBUG [Listener at localhost/42999] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-08 07:53:35,886 INFO [Listener at localhost/42999] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35671 2023-03-08 07:53:35,896 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:35,898 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:35,919 INFO [Listener at localhost/42999] zookeeper.RecoverableZooKeeper(93): Process identifier=master:35671 connecting to ZooKeeper ensemble=127.0.0.1:52387 2023-03-08 07:53:35,958 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:356710x0, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 07:53:35,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:35671-0x100c31dfa1f0000 connected 2023-03-08 07:53:35,991 DEBUG [Listener at localhost/42999] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 07:53:35,992 DEBUG [Listener at localhost/42999] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 07:53:35,995 DEBUG [Listener at localhost/42999] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 07:53:36,003 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35671 2023-03-08 07:53:36,003 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35671 2023-03-08 07:53:36,003 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35671 2023-03-08 07:53:36,004 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35671 2023-03-08 07:53:36,004 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35671 2023-03-08 07:53:36,010 INFO [Listener at localhost/42999] master.HMaster(444): hbase.rootdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b, hbase.cluster.distributed=false 2023-03-08 07:53:36,085 INFO [Listener at localhost/42999] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-08 07:53:36,085 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 07:53:36,085 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 07:53:36,086 INFO [Listener at localhost/42999] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 07:53:36,086 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 07:53:36,086 INFO [Listener at localhost/42999] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 07:53:36,093 INFO [Listener at localhost/42999] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 07:53:36,097 INFO [Listener at localhost/42999] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33669 2023-03-08 07:53:36,100 INFO [Listener at localhost/42999] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-08 07:53:36,106 DEBUG [Listener at localhost/42999] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-08 07:53:36,106 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:36,109 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:36,111 INFO [Listener at localhost/42999] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33669 connecting to ZooKeeper ensemble=127.0.0.1:52387 2023-03-08 07:53:36,115 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:336690x0, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 07:53:36,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33669-0x100c31dfa1f0001 connected 2023-03-08 07:53:36,116 DEBUG [Listener at localhost/42999] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 07:53:36,117 DEBUG [Listener at localhost/42999] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 07:53:36,118 DEBUG [Listener at localhost/42999] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 07:53:36,122 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33669 2023-03-08 07:53:36,122 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33669 2023-03-08 07:53:36,124 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33669 2023-03-08 07:53:36,124 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33669 2023-03-08 07:53:36,128 DEBUG [Listener at localhost/42999] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33669 2023-03-08 07:53:36,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:36,138 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 07:53:36,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:36,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 07:53:36,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 07:53:36,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:36,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 07:53:36,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,35671,1678262014900 from backup master directory 2023-03-08 07:53:36,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 07:53:36,164 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:36,164 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 07:53:36,165 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 07:53:36,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:36,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-08 07:53:36,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-08 07:53:36,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/hbase.id with ID: 77b17b02-4516-4bcb-91a4-af656134e496 2023-03-08 07:53:36,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:36,314 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:36,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3aefaa6e to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:53:36,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e77887a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:53:36,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-08 07:53:36,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-08 07:53:36,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-08 07:53:36,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-08 07:53:36,436 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 07:53:36,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 07:53:36,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 07:53:36,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store-tmp 2023-03-08 07:53:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 07:53:36,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:53:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:53:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 07:53:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:53:36,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:53:36,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 07:53:36,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/WALs/jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:36,937 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35671%2C1678262014900, suffix=, logDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/WALs/jenkins-hbase4.apache.org,35671,1678262014900, archiveDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/oldWALs, maxLogs=10 2023-03-08 07:53:36,998 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40571,DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321,DISK] 2023-03-08 07:53:37,006 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-08 07:53:37,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/WALs/jenkins-hbase4.apache.org,35671,1678262014900/jenkins-hbase4.apache.org%2C35671%2C1678262014900.1678262016948 2023-03-08 07:53:37,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40571,DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321,DISK]] 2023-03-08 07:53:37,072 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:53:37,072 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:37,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 07:53:37,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 07:53:37,139 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-08 07:53:37,146 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-08 07:53:37,170 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-08 07:53:37,183 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:37,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 07:53:37,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 07:53:37,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 07:53:37,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:53:37,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70886574, jitterRate=0.05629226565361023}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:53:37,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 07:53:37,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-08 07:53:37,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-08 07:53:37,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-08 07:53:37,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-08 07:53:37,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-08 07:53:37,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-03-08 07:53:37,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-08 07:53:37,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-08 07:53:37,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-08 07:53:37,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-08 07:53:37,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-08 07:53:37,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-08 07:53:37,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-08 07:53:37,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-08 07:53:37,378 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:37,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-08 07:53:37,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-08 07:53:37,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-08 07:53:37,401 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 07:53:37,401 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 07:53:37,401 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:37,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,35671,1678262014900, sessionid=0x100c31dfa1f0000, setting cluster-up flag (Was=false) 2023-03-08 07:53:37,421 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:37,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-08 07:53:37,428 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:37,437 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:37,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-08 07:53:37,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:37,448 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp 2023-03-08 07:53:37,532 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(952): ClusterId : 77b17b02-4516-4bcb-91a4-af656134e496 2023-03-08 07:53:37,538 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-08 07:53:37,544 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-08 07:53:37,544 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-08 07:53:37,548 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-08 07:53:37,548 DEBUG [RS:0;jenkins-hbase4:33669] zookeeper.ReadOnlyZKClient(139): Connect 0x749237b6 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:53:37,567 DEBUG [RS:0;jenkins-hbase4:33669] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a55d754, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:53:37,567 DEBUG [RS:0;jenkins-hbase4:33669] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e6a5cc6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-08 07:53:37,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-08 07:53:37,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 07:53:37,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 07:53:37,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 07:53:37,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 07:53:37,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-08 07:53:37,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-08 07:53:37,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,600 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33669 2023-03-08 07:53:37,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678262047604 2023-03-08 07:53:37,606 INFO [RS:0;jenkins-hbase4:33669] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-08 07:53:37,606 INFO [RS:0;jenkins-hbase4:33669] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-08 07:53:37,606 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1023): About to register with Master. 2023-03-08 07:53:37,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-08 07:53:37,610 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 07:53:37,610 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-08 07:53:37,612 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35671,1678262014900 with isa=jenkins-hbase4.apache.org/172.31.14.131:33669, startcode=1678262016084 2023-03-08 07:53:37,617 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 07:53:37,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-08 07:53:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-08 07:53:37,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-08 07:53:37,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-08 07:53:37,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-08 07:53:37,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-08 07:53:37,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-08 07:53:37,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-08 07:53:37,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-08 07:53:37,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-08 07:53:37,638 DEBUG [RS:0;jenkins-hbase4:33669] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-08 07:53:37,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678262017639,5,FailOnTimeoutGroup] 2023-03-08 07:53:37,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678262017640,5,FailOnTimeoutGroup] 2023-03-08 07:53:37,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-08 07:53:37,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,700 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57817, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-08 07:53:37,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:37,730 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b 2023-03-08 07:53:37,730 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:39077 2023-03-08 07:53:37,730 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-08 07:53:37,737 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 07:53:37,738 DEBUG [RS:0;jenkins-hbase4:33669] zookeeper.ZKUtil(162): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:37,738 WARN [RS:0;jenkins-hbase4:33669] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 07:53:37,739 INFO [RS:0;jenkins-hbase4:33669] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 07:53:37,739 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1947): logDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/WALs/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:37,740 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33669,1678262016084] 2023-03-08 07:53:37,752 DEBUG [RS:0;jenkins-hbase4:33669] zookeeper.ZKUtil(162): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:37,764 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-08 07:53:37,774 INFO [RS:0;jenkins-hbase4:33669] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-08 07:53:37,805 INFO [RS:0;jenkins-hbase4:33669] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-08 07:53:37,809 INFO [RS:0;jenkins-hbase4:33669] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:53:37,809 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,810 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-08 07:53:37,818 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,819 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,819 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,819 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,820 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,820 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,820 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-08 07:53:37,820 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,820 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,820 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,821 DEBUG [RS:0;jenkins-hbase4:33669] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 07:53:37,822 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,822 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,822 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,840 INFO [RS:0;jenkins-hbase4:33669] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-08 07:53:37,843 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33669,1678262016084-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:37,864 INFO [RS:0;jenkins-hbase4:33669] regionserver.Replication(203): jenkins-hbase4.apache.org,33669,1678262016084 started 2023-03-08 07:53:37,864 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33669,1678262016084, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33669, sessionid=0x100c31dfa1f0001 2023-03-08 07:53:37,864 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-08 07:53:37,864 DEBUG [RS:0;jenkins-hbase4:33669] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:37,865 DEBUG [RS:0;jenkins-hbase4:33669] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33669,1678262016084' 2023-03-08 07:53:37,865 DEBUG [RS:0;jenkins-hbase4:33669] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:37,866 DEBUG [RS:0;jenkins-hbase4:33669] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:37,866 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-08 07:53:37,867 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-08 07:53:37,867 DEBUG [RS:0;jenkins-hbase4:33669] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:37,867 DEBUG [RS:0;jenkins-hbase4:33669] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33669,1678262016084' 2023-03-08 07:53:37,867 DEBUG [RS:0;jenkins-hbase4:33669] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 07:53:37,867 DEBUG [RS:0;jenkins-hbase4:33669] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 07:53:37,868 DEBUG [RS:0;jenkins-hbase4:33669] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-08 07:53:37,868 INFO [RS:0;jenkins-hbase4:33669] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-08 07:53:37,868 INFO [RS:0;jenkins-hbase4:33669] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-08 07:53:37,981 INFO [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33669%2C1678262016084, suffix=, logDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/WALs/jenkins-hbase4.apache.org,33669,1678262016084, archiveDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/oldWALs, maxLogs=32 2023-03-08 07:53:38,000 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40571,DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321,DISK] 2023-03-08 07:53:38,004 INFO [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/WALs/jenkins-hbase4.apache.org,33669,1678262016084/jenkins-hbase4.apache.org%2C33669%2C1678262016084.1678262017983 2023-03-08 07:53:38,004 DEBUG [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40571,DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321,DISK]] 2023-03-08 07:53:38,057 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 07:53:38,058 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 07:53:38,058 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b 2023-03-08 07:53:38,472 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:38,475 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 07:53:38,477 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/info 2023-03-08 07:53:38,479 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 07:53:38,480 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:38,480 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 07:53:38,483 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/rep_barrier 2023-03-08 07:53:38,484 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 07:53:38,485 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:38,485 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 07:53:38,488 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/table 2023-03-08 07:53:38,488 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 07:53:38,489 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:38,490 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740 2023-03-08 07:53:38,491 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740 2023-03-08 07:53:38,495 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 07:53:38,497 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 07:53:38,501 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:53:38,502 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74675757, jitterRate=0.11275549232959747}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 07:53:38,502 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 07:53:38,502 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 07:53:38,502 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 07:53:38,502 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 07:53:38,502 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 07:53:38,502 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 07:53:38,503 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 07:53:38,503 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 07:53:38,509 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 07:53:38,509 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-08 07:53:38,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-08 07:53:38,528 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-08 07:53:38,530 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-08 07:53:38,682 DEBUG [jenkins-hbase4:35671] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-08 07:53:38,686 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33669,1678262016084, state=OPENING 2023-03-08 07:53:38,693 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-08 07:53:38,695 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:38,695 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 07:53:38,698 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:53:38,882 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:38,885 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 07:53:38,888 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46252, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 07:53:38,898 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-08 07:53:38,899 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 07:53:38,902 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33669%2C1678262016084.meta, suffix=.meta, logDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/WALs/jenkins-hbase4.apache.org,33669,1678262016084, archiveDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/oldWALs, maxLogs=32 2023-03-08 07:53:38,919 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40571,DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321,DISK] 2023-03-08 07:53:38,924 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/WALs/jenkins-hbase4.apache.org,33669,1678262016084/jenkins-hbase4.apache.org%2C33669%2C1678262016084.meta.1678262018904.meta 2023-03-08 07:53:38,924 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40571,DS-ea36e849-35f2-4f15-9fce-95ac4f9f2321,DISK]] 2023-03-08 07:53:38,925 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:53:38,926 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-08 07:53:38,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-08 07:53:38,949 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-08 07:53:38,955 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-08 07:53:38,955 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:38,955 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-08 07:53:38,955 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-08 07:53:38,958 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 07:53:38,960 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/info 2023-03-08 07:53:38,960 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/info 2023-03-08 07:53:38,960 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 07:53:38,961 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:38,961 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 07:53:38,963 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/rep_barrier 2023-03-08 07:53:38,963 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/rep_barrier 2023-03-08 07:53:38,963 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 07:53:38,964 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:38,964 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 07:53:38,965 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/table 2023-03-08 07:53:38,966 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/table 2023-03-08 07:53:38,966 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 07:53:38,967 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:38,968 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740 2023-03-08 07:53:38,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740 2023-03-08 07:53:38,974 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 07:53:38,976 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 07:53:38,977 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63974278, jitterRate=-0.04670897126197815}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 07:53:38,977 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 07:53:38,986 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678262018875 2023-03-08 07:53:39,004 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-08 07:53:39,004 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-08 07:53:39,005 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33669,1678262016084, state=OPEN 2023-03-08 07:53:39,008 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-08 07:53:39,008 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 07:53:39,012 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-08 07:53:39,012 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33669,1678262016084 in 310 msec 2023-03-08 07:53:39,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-08 07:53:39,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 497 msec 2023-03-08 07:53:39,022 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5190 sec 2023-03-08 07:53:39,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678262019023, completionTime=-1 2023-03-08 07:53:39,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-08 07:53:39,023 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-08 07:53:39,083 DEBUG [hconnection-0x780ddb73-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:53:39,087 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46266, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:53:39,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-08 07:53:39,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678262079106 2023-03-08 07:53:39,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678262139106 2023-03-08 07:53:39,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 83 msec 2023-03-08 07:53:39,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35671,1678262014900-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:39,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35671,1678262014900-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:39,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35671,1678262014900-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:39,144 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:35671, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:39,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-08 07:53:39,153 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-08 07:53:39,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-08 07:53:39,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 07:53:39,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-08 07:53:39,174 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:53:39,176 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:53:39,196 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,198 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf empty. 2023-03-08 07:53:39,199 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,199 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-08 07:53:39,220 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-08 07:53:39,223 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9b26a0fbb0144612b31a0a0058f9f1bf, NAME => 'hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:53:39,236 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:39,236 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 9b26a0fbb0144612b31a0a0058f9f1bf, disabling compactions & flushes 2023-03-08 07:53:39,236 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,236 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,236 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. after waiting 0 ms 2023-03-08 07:53:39,236 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,236 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,236 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 9b26a0fbb0144612b31a0a0058f9f1bf: 2023-03-08 07:53:39,243 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:53:39,260 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678262019248"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262019248"}]},"ts":"1678262019248"} 2023-03-08 07:53:39,300 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:53:39,303 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:53:39,308 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262019303"}]},"ts":"1678262019303"} 2023-03-08 07:53:39,313 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-08 07:53:39,327 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=9b26a0fbb0144612b31a0a0058f9f1bf, ASSIGN}] 2023-03-08 07:53:39,330 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=9b26a0fbb0144612b31a0a0058f9f1bf, ASSIGN 2023-03-08 07:53:39,331 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=9b26a0fbb0144612b31a0a0058f9f1bf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:53:39,482 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=9b26a0fbb0144612b31a0a0058f9f1bf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:39,483 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678262019482"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262019482"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262019482"}]},"ts":"1678262019482"} 2023-03-08 07:53:39,490 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 9b26a0fbb0144612b31a0a0058f9f1bf, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:53:39,650 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,651 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9b26a0fbb0144612b31a0a0058f9f1bf, NAME => 'hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:53:39,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:39,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,654 INFO [StoreOpener-9b26a0fbb0144612b31a0a0058f9f1bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,656 DEBUG [StoreOpener-9b26a0fbb0144612b31a0a0058f9f1bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/info 2023-03-08 07:53:39,656 DEBUG [StoreOpener-9b26a0fbb0144612b31a0a0058f9f1bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/info 2023-03-08 07:53:39,657 INFO [StoreOpener-9b26a0fbb0144612b31a0a0058f9f1bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b26a0fbb0144612b31a0a0058f9f1bf columnFamilyName info 2023-03-08 07:53:39,657 INFO [StoreOpener-9b26a0fbb0144612b31a0a0058f9f1bf-1] regionserver.HStore(310): Store=9b26a0fbb0144612b31a0a0058f9f1bf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:39,659 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,660 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,664 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:53:39,667 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:53:39,668 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9b26a0fbb0144612b31a0a0058f9f1bf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68130200, jitterRate=0.01521909236907959}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:53:39,668 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9b26a0fbb0144612b31a0a0058f9f1bf: 2023-03-08 07:53:39,670 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf., pid=6, masterSystemTime=1678262019644 2023-03-08 07:53:39,674 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,674 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:53:39,676 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=9b26a0fbb0144612b31a0a0058f9f1bf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:39,676 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678262019675"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262019675"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262019675"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262019675"}]},"ts":"1678262019675"} 2023-03-08 07:53:39,683 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-08 07:53:39,684 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 9b26a0fbb0144612b31a0a0058f9f1bf, server=jenkins-hbase4.apache.org,33669,1678262016084 in 190 msec 2023-03-08 07:53:39,688 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-08 07:53:39,689 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=9b26a0fbb0144612b31a0a0058f9f1bf, ASSIGN in 356 msec 2023-03-08 07:53:39,690 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:53:39,691 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262019690"}]},"ts":"1678262019690"} 2023-03-08 07:53:39,694 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-08 07:53:39,697 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:53:39,700 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 535 msec 2023-03-08 07:53:39,774 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-08 07:53:39,777 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-08 07:53:39,777 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:53:39,816 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-08 07:53:39,832 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 07:53:39,837 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-03-08 07:53:39,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-08 07:53:39,864 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 07:53:39,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-03-08 07:53:39,879 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-08 07:53:39,882 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-08 07:53:39,883 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.717sec 2023-03-08 07:53:39,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-08 07:53:39,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-08 07:53:39,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-08 07:53:39,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35671,1678262014900-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-08 07:53:39,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35671,1678262014900-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-08 07:53:39,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-08 07:53:39,939 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(139): Connect 0x57919e99 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:53:39,945 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73d84839, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:53:39,958 DEBUG [hconnection-0x346dc13e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:53:39,973 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46278, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:53:39,984 INFO [Listener at localhost/42999] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:53:39,985 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(139): Connect 0x480b078d to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:53:39,989 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15776577, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:53:39,990 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:53:40,014 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=401, ProcessCount=170, AvailableMemoryMB=7802 2023-03-08 07:53:40,025 DEBUG [Listener at localhost/42999] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:53:40,028 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32910, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:53:40,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 07:53:40,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:40,044 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:53:40,046 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:53:40,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-08 07:53:40,050 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,051 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c empty. 2023-03-08 07:53:40,052 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,052 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-08 07:53:40,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 07:53:40,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 07:53:40,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 07:53:40,471 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 07:53:40,473 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 38efdc462c389fa50e647861e0aace4c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:53:40,486 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:40,486 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 38efdc462c389fa50e647861e0aace4c, disabling compactions & flushes 2023-03-08 07:53:40,487 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,487 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,487 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. after waiting 0 ms 2023-03-08 07:53:40,487 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,487 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,487 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:40,493 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:53:40,495 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262020494"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262020494"}]},"ts":"1678262020494"} 2023-03-08 07:53:40,498 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:53:40,500 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:53:40,500 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262020500"}]},"ts":"1678262020500"} 2023-03-08 07:53:40,503 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-08 07:53:40,508 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, ASSIGN}] 2023-03-08 07:53:40,511 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, ASSIGN 2023-03-08 07:53:40,512 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:53:40,664 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=38efdc462c389fa50e647861e0aace4c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:40,664 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262020664"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262020664"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262020664"}]},"ts":"1678262020664"} 2023-03-08 07:53:40,668 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 38efdc462c389fa50e647861e0aace4c, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:53:40,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 07:53:40,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 38efdc462c389fa50e647861e0aace4c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:53:40,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:40,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,830 INFO [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,832 DEBUG [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:40,833 DEBUG [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:40,836 INFO [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38efdc462c389fa50e647861e0aace4c columnFamilyName family1 2023-03-08 07:53:40,839 INFO [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] regionserver.HStore(310): Store=38efdc462c389fa50e647861e0aace4c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:40,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:40,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:53:40,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 38efdc462c389fa50e647861e0aace4c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69700263, jitterRate=0.03861485421657562}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:53:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:40,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., pid=11, masterSystemTime=1678262020821 2023-03-08 07:53:40,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:40,857 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=38efdc462c389fa50e647861e0aace4c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:40,857 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262020857"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262020857"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262020857"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262020857"}]},"ts":"1678262020857"} 2023-03-08 07:53:40,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-08 07:53:40,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 38efdc462c389fa50e647861e0aace4c, server=jenkins-hbase4.apache.org,33669,1678262016084 in 193 msec 2023-03-08 07:53:40,866 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-08 07:53:40,867 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, ASSIGN in 355 msec 2023-03-08 07:53:40,868 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:53:40,868 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262020868"}]},"ts":"1678262020868"} 2023-03-08 07:53:40,870 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-08 07:53:40,873 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:53:40,875 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 832 msec 2023-03-08 07:53:41,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 07:53:41,171 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-08 07:53:41,194 DEBUG [hconnection-0x244f457c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:53:41,197 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46294, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:53:41,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:41,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,382 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:41,382 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:41,383 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:41,383 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:41,384 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,384 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,386 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:41,386 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:41,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:41,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:41,386 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:41,386 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:41,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:41,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,388 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:41,391 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:41,391 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:41,392 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:41,393 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:41,393 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:41,393 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:41,394 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:41,395 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:53:41,981 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300d3bd2f0009c495cb602ef03dc0399c8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300d3bd2f0009c495cb602ef03dc0399c8 2023-03-08 07:53:42,398 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/e2c16c94ac094a0fb6d80980812c898a 2023-03-08 07:53:42,408 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/e2c16c94ac094a0fb6d80980812c898a as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/e2c16c94ac094a0fb6d80980812c898a 2023-03-08 07:53:42,418 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/e2c16c94ac094a0fb6d80980812c898a, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-08 07:53:42,420 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 1025ms, sequenceid=5, compaction requested=false 2023-03-08 07:53:42,422 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:42,424 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:42,425 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:42,428 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,428 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:42,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:42,428 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,428 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:42,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:42,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:42,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:42,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:42,430 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:42,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6dc4d60a[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:42,430 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,432 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,432 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,432 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:42,432 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:42,432 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:42,432 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:42,434 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:42,434 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:42,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:42,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:42,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:42,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:42,438 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:42,438 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:42,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:42,438 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:42,438 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,438 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:42,442 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,442 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,442 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:42,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:42,444 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:42,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:42,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:42,451 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:42,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,452 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:42,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:42,454 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:42,454 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:42,554 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:42,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:42,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:42,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:42,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,571 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:42,571 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:42,572 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:42,572 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:42,572 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,572 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:42,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:42,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:42,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:42,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:42,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:42,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:42,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:42,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:42,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:42,577 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:42,577 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:42,577 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:42,577 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:42,577 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:53:43,002 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289ff923bac42f4ded8330bf681995a124 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289ff923bac42f4ded8330bf681995a124 2023-03-08 07:53:43,414 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/d6a4ef896044462383852adc29ae8b0e 2023-03-08 07:53:43,424 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/d6a4ef896044462383852adc29ae8b0e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/d6a4ef896044462383852adc29ae8b0e 2023-03-08 07:53:43,432 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/d6a4ef896044462383852adc29ae8b0e, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:53:43,434 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 857ms, sequenceid=9, compaction requested=false 2023-03-08 07:53:43,435 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:43,435 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:43,437 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,437 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:43,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:43,438 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,438 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:43,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:43,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71abf63c[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:43,440 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:43,440 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,442 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:43,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:43,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:43,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:43,444 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:43,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,444 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:43,444 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:43,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:43,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:43,448 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:43,448 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:43,448 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:43,448 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,448 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:43,453 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,453 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,453 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:43,454 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:43,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,455 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,460 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,460 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:43,460 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:43,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:43,460 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:43,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:43,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:43,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:43,460 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:43,460 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,461 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,461 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,461 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:43,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:43,461 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,461 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:43,461 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:43,561 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:43,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:43,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:43,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:43,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,573 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:43,573 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:43,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:43,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:43,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,576 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:43,576 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:43,576 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:43,576 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:43,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:43,578 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:43,578 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:43,578 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:43,579 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:43,579 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:43,579 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:43,579 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:43,579 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:43,770 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-08 07:53:43,771 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-08 07:53:44,007 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201585b9f1c105c4753a19e3bf507a78754 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201585b9f1c105c4753a19e3bf507a78754 2023-03-08 07:53:44,422 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/5fcb5dcc907b4e51ac485a54976989d8 2023-03-08 07:53:44,432 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/5fcb5dcc907b4e51ac485a54976989d8 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5fcb5dcc907b4e51ac485a54976989d8 2023-03-08 07:53:44,440 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5fcb5dcc907b4e51ac485a54976989d8, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 07:53:44,441 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 862ms, sequenceid=13, compaction requested=true 2023-03-08 07:53:44,441 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:44,441 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:44,441 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:44,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:44,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:44,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:44,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,444 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:44,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:44,445 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,445 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:44,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:44,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:44,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:44,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:44,447 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68a6ccc8[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:44,447 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:44,447 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,448 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,448 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:44,448 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:44,448 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:44,448 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,448 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:44,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:44,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:44,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:44,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:44,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:44,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:44,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:44,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:44,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:44,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:44,454 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:44,454 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:44,454 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:44,454 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,455 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:44,456 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,456 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:44,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,456 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:44,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,458 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:44,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,459 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:44,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:44,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:44,463 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:44,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:44,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:44,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,463 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:44,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:44,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:44,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:44,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:44,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,465 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:44,465 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:44,490 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:53:44,565 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:44,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:44,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:44,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:44,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,578 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:44,578 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:44,578 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:44,578 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:44,579 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,579 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,581 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:44,581 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:44,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:44,581 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:44,581 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:44,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:44,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:44,583 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:44,583 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:44,584 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:44,584 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:44,584 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:44,584 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:44,584 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:44,584 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:45,014 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057ff8708ddf6f4236a3959a58774bd97e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057ff8708ddf6f4236a3959a58774bd97e 2023-03-08 07:53:45,061 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 07:53:45,061 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-08 07:53:45,427 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/cc828dc770c64a02ba19d41f7ebc6703 2023-03-08 07:53:45,436 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/cc828dc770c64a02ba19d41f7ebc6703 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/cc828dc770c64a02ba19d41f7ebc6703 2023-03-08 07:53:45,445 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/cc828dc770c64a02ba19d41f7ebc6703, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 07:53:45,446 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 862ms, sequenceid=17, compaction requested=true 2023-03-08 07:53:45,447 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:45,447 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:45,452 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,452 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:45,453 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,453 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:45,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:45,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ac88b9d[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:45,455 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:45,455 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,456 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:45,456 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,456 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:45,458 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:45,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:45,462 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:45,462 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:45,462 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:45,462 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,462 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:45,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:45,463 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:45,464 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,466 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:45,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,471 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:45,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:45,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:45,472 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:45,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:45,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:45,472 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,473 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:45,473 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:45,473 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:45,574 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:45,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:45,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:45,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:45,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,586 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:45,586 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:45,587 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:45,587 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:45,587 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,587 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,589 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,589 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:45,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:45,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:45,589 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:45,589 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:45,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:45,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:45,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:45,591 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:45,591 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:45,591 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:45,591 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:45,591 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:45,591 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:45,591 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:45,591 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:46,016 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283c76d9b604ab4ab6b6c6ea346e64ff86 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283c76d9b604ab4ab6b6c6ea346e64ff86 2023-03-08 07:53:46,429 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/a1829bb82f864f5fb3b10e88659b09b3 2023-03-08 07:53:46,440 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/a1829bb82f864f5fb3b10e88659b09b3 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a1829bb82f864f5fb3b10e88659b09b3 2023-03-08 07:53:46,449 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a1829bb82f864f5fb3b10e88659b09b3, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 07:53:46,451 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 860ms, sequenceid=21, compaction requested=true 2023-03-08 07:53:46,451 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:46,451 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:46,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:46,452 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:46,452 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,452 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:46,452 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:46,457 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,457 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:46,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:46,458 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,458 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:46,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:46,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:46,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:46,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:46,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51f77a48[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:46,460 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:46,460 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,462 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,463 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:46,463 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:46,463 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,463 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:46,463 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:46,464 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:46,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,464 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:46,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:46,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:46,464 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:46,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:46,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:46,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:46,468 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:46,468 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:46,468 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:46,468 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,468 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:46,470 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,470 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:46,470 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:46,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:46,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:46,471 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:46,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:46,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,473 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:46,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:46,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:46,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:46,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:46,476 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:46,476 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,477 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:46,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:46,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:46,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:46,478 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:46,478 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:46,578 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:46,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:46,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:46,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:46,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,590 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:46,591 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:46,591 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:46,591 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:46,592 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,592 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,593 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,593 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:46,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:46,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:46,593 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:46,594 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:46,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:46,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:46,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:46,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:46,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:46,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:46,596 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:46,596 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:46,596 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:46,596 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:46,596 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:46,626 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317042c94038e94c11b1adae8d1e397687 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317042c94038e94c11b1adae8d1e397687 2023-03-08 07:53:47,040 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/dadf754747ec4bc18649ec3fd6305da1 2023-03-08 07:53:47,050 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/dadf754747ec4bc18649ec3fd6305da1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dadf754747ec4bc18649ec3fd6305da1 2023-03-08 07:53:47,059 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dadf754747ec4bc18649ec3fd6305da1, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 07:53:47,060 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 464ms, sequenceid=25, compaction requested=true 2023-03-08 07:53:47,060 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:47,060 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:47,060 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:47,061 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:47,061 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,061 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:47,061 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:47,064 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,064 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:47,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f437c5c[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:47,069 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:47,069 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,070 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:47,071 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,071 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:47,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:47,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:47,072 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:47,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:47,077 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:47,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:47,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:47,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,077 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:47,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:47,080 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,086 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,090 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,090 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:47,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:47,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:47,090 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:47,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:47,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:47,090 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:47,090 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,091 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,091 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,091 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,091 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,091 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:47,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:47,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:47,099 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:47,099 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:47,199 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:47,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:47,210 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:47,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:47,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,213 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:47,213 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:47,214 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:47,214 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:47,214 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,214 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,216 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:47,216 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:47,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:47,216 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,216 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:47,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:47,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,220 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:47,220 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:47,221 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:47,224 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:47,224 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:47,224 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:47,224 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:47,224 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:47,279 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101141d58f8284e40b0868e37cd039b8302 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101141d58f8284e40b0868e37cd039b8302 2023-03-08 07:53:47,299 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/5f0b8a4cc77c484cae3d164e165865ee 2023-03-08 07:53:47,310 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/5f0b8a4cc77c484cae3d164e165865ee as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5f0b8a4cc77c484cae3d164e165865ee 2023-03-08 07:53:47,322 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5f0b8a4cc77c484cae3d164e165865ee, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 07:53:47,323 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 99ms, sequenceid=29, compaction requested=true 2023-03-08 07:53:47,324 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:47,324 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:47,324 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:47,325 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:47,325 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,325 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:47,325 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:47,328 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,328 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,329 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,329 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:47,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:47,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@215d2ff0[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:47,333 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:47,333 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,335 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:47,335 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,335 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:47,341 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:47,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:47,346 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:47,346 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:47,346 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:47,346 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,346 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:47,349 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,349 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,349 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,350 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,352 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,357 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:47,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:47,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:47,357 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:47,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:47,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:47,358 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,359 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,359 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:47,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:47,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:47,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:47,459 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:47,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:47,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:47,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:47,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,483 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:47,483 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:47,483 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:47,484 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:47,484 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,484 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,486 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,486 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:47,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:47,486 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,486 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,487 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:47,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,488 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:47,488 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:47,488 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:47,492 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:47,492 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:47,492 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:47,547 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038591061cd1654c1a8f1014aa90ddc20a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038591061cd1654c1a8f1014aa90ddc20a 2023-03-08 07:53:47,961 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/3e4f8f35a1c547158d9a319e6a5c3cb0 2023-03-08 07:53:47,969 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/3e4f8f35a1c547158d9a319e6a5c3cb0 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/3e4f8f35a1c547158d9a319e6a5c3cb0 2023-03-08 07:53:47,976 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/3e4f8f35a1c547158d9a319e6a5c3cb0, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 07:53:47,978 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 486ms, sequenceid=33, compaction requested=true 2023-03-08 07:53:47,978 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:47,978 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:47,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:47,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:47,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:47,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:47,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,980 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,981 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,981 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:47,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f97dc25[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:47,983 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:47,983 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,984 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:47,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:47,985 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:47,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:47,990 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:47,990 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:47,990 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:47,990 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,990 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:47,992 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,992 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,992 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:47,993 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,994 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,997 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,997 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,997 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:47,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:47,997 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:47,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 07:53:47,997 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:47,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,997 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:47,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:47,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:47,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:47,998 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:47,999 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:47,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:48,099 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:48,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:48,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:48,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:48,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,109 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:48,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:48,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:48,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:48,111 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,111 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,112 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:48,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:48,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:48,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:48,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,113 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:48,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:48,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:48,115 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:48,115 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:48,115 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:48,115 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:48,115 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:48,115 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:48,535 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907415ae2d81d2644418a81dc8fcbe091c8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907415ae2d81d2644418a81dc8fcbe091c8 2023-03-08 07:53:48,944 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/0ebf25f719cc439ea59f6fc6626902cb 2023-03-08 07:53:48,952 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/0ebf25f719cc439ea59f6fc6626902cb as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/0ebf25f719cc439ea59f6fc6626902cb 2023-03-08 07:53:48,958 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/0ebf25f719cc439ea59f6fc6626902cb, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 07:53:48,959 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 844ms, sequenceid=37, compaction requested=true 2023-03-08 07:53:48,960 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:48,960 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,962 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:48,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:48,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1558cca5[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:48,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:48,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,966 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,966 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:48,967 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:48,967 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:48,967 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,967 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:48,968 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:48,968 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,968 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:48,968 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:48,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:48,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:48,971 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:48,971 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:48,971 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:48,971 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,971 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:48,972 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,972 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:48,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:48,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,972 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:48,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:48,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:48,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:48,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,974 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:48,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:48,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:48,977 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:48,977 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:48,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:48,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:48,976 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:48,977 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,977 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,977 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:48,978 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:48,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:48,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:48,978 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:49,078 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:49,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:49,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:49,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:49,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,090 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:49,090 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:49,091 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 07:53:49,091 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 07:53:49,091 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,091 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,094 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,094 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:49,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:49,094 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,094 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 07:53:49,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 07:53:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 07:53:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:49,098 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:49,098 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. started... 2023-03-08 07:53:49,098 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38efdc462c389fa50e647861e0aace4c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:49,522 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b938e565ed5e490788a7395caf35fddc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120b938e565ed5e490788a7395caf35fddc 2023-03-08 07:53:49,957 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/a259443066e147689031a2d4aa2f37e0 2023-03-08 07:53:49,964 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/a259443066e147689031a2d4aa2f37e0 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a259443066e147689031a2d4aa2f37e0 2023-03-08 07:53:49,970 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a259443066e147689031a2d4aa2f37e0, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 07:53:49,971 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 38efdc462c389fa50e647861e0aace4c in 873ms, sequenceid=41, compaction requested=true 2023-03-08 07:53:49,971 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:49,971 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:49,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:49,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 07:53:49,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 07:53:49,974 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:49,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 07:53:49,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@711de3ce[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:49,976 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 07:53:49,976 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,977 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,977 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 07:53:49,977 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 07:53:49,977 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:49,977 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,978 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:49,987 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 07:53:49,987 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,987 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:49,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:49,987 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 07:53:49,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:49,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:49,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:49,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:49,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 07:53:49,991 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 07:53:49,991 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:49,991 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:49,991 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,991 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:49,993 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,993 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:49,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,993 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:49,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:49,994 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:49,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,995 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:49,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:49,998 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:49,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 07:53:49,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:49,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,998 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:49,999 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:49,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:49,999 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:50,099 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:50,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:50,106 DEBUG [Listener at localhost/42999] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 07:53:50,109 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36994, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 07:53:50,110 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:50,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 07:53:50,127 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 07:53:50,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:53:50,128 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262630128 2023-03-08 07:53:50,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:53:50,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 07:53:50,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:50,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x59d158ad to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:53:50,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@273a58a2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:53:50,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,607 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010388115834dc404f629f3bb9aecd662e89 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010388115834dc404f629f3bb9aecd662e89 2023-03-08 07:53:50,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:50,626 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:53:50,628 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45560, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:53:50,639 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:53:50,641 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37010, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:53:50,676 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/235e4577cb0e4b15b046b338fdb6ebd9 first=Optional[r3] last=Optional[r6] 2023-03-08 07:53:50,689 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/235e4577cb0e4b15b046b338fdb6ebd9}] 2023-03-08 07:53:50,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/235e4577cb0e4b15b046b338fdb6ebd9 for inclusion in 38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:50,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 07:53:50,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:53:50,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:50,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/235e4577cb0e4b15b046b338fdb6ebd9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__708f0iop0ek06nkirjc91ina59eo6ohte05viccufh89s5k51hap4kia2hj3ldo0/family1/235e4577cb0e4b15b046b338fdb6ebd9 2023-03-08 07:53:50,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__708f0iop0ek06nkirjc91ina59eo6ohte05viccufh89s5k51hap4kia2hj3ldo0/family1/235e4577cb0e4b15b046b338fdb6ebd9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_ 2023-03-08 07:53:50,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__708f0iop0ek06nkirjc91ina59eo6ohte05viccufh89s5k51hap4kia2hj3ldo0/family1/235e4577cb0e4b15b046b338fdb6ebd9 into 38efdc462c389fa50e647861e0aace4c/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_ - updating store file list. 2023-03-08 07:53:50,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_ into 38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:50,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__708f0iop0ek06nkirjc91ina59eo6ohte05viccufh89s5k51hap4kia2hj3ldo0/family1/235e4577cb0e4b15b046b338fdb6ebd9 into 38efdc462c389fa50e647861e0aace4c/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_) 2023-03-08 07:53:50,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__708f0iop0ek06nkirjc91ina59eo6ohte05viccufh89s5k51hap4kia2hj3ldo0/family1/235e4577cb0e4b15b046b338fdb6ebd9 2023-03-08 07:53:50,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:53:50,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:53:50,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 38efdc462c389fa50e647861e0aace4c/family1 is initiating minor compaction 2023-03-08 07:53:50,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5b4c116; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:53:50,735 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 38efdc462c389fa50e647861e0aace4c/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:50,735 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/d6a4ef896044462383852adc29ae8b0e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5fcb5dcc907b4e51ac485a54976989d8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/cc828dc770c64a02ba19d41f7ebc6703, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a1829bb82f864f5fb3b10e88659b09b3, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dadf754747ec4bc18649ec3fd6305da1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5f0b8a4cc77c484cae3d164e165865ee, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/3e4f8f35a1c547158d9a319e6a5c3cb0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/0ebf25f719cc439ea59f6fc6626902cb, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a259443066e147689031a2d4aa2f37e0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp, totalSize=50.2 K 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting d6a4ef896044462383852adc29ae8b0e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 5fcb5dcc907b4e51ac485a54976989d8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting cc828dc770c64a02ba19d41f7ebc6703, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting a1829bb82f864f5fb3b10e88659b09b3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting dadf754747ec4bc18649ec3fd6305da1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 5f0b8a4cc77c484cae3d164e165865ee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 3e4f8f35a1c547158d9a319e6a5c3cb0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 0ebf25f719cc439ea59f6fc6626902cb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting a259443066e147689031a2d4aa2f37e0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:53:50,736 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 5106ccdc82bd44e2840b1c3a94433450_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:53:50,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:50,751 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:53:50,756 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283c76d9b604ab4ab6b6c6ea346e64ff86 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283c76d9b604ab4ab6b6c6ea346e64ff86 2023-03-08 07:53:50,758 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317042c94038e94c11b1adae8d1e397687 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317042c94038e94c11b1adae8d1e397687 2023-03-08 07:53:50,760 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101141d58f8284e40b0868e37cd039b8302 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101141d58f8284e40b0868e37cd039b8302 2023-03-08 07:53:50,761 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038591061cd1654c1a8f1014aa90ddc20a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038591061cd1654c1a8f1014aa90ddc20a 2023-03-08 07:53:50,761 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-08 07:53:50,761 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 07:53:50,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,763 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,763 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:50,789 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 38efdc462c389fa50e647861e0aace4c#family1#compaction#10 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:53:51,615 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/26d22426baf84eb9b47dffc610db496b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/26d22426baf84eb9b47dffc610db496b 2023-03-08 07:53:51,631 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 38efdc462c389fa50e647861e0aace4c/family1 of 38efdc462c389fa50e647861e0aace4c into 26d22426baf84eb9b47dffc610db496b(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:53:51,631 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:51,632 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678262030733; duration=0sec 2023-03-08 07:53:51,633 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:53:51,647 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120533db9354ce944a90bf99858e273f3a2f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120533db9354ce944a90bf99858e273f3a2f 2023-03-08 07:53:51,649 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:51,681 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8a36d79128d34588858cdbd3758340af first=Optional[r00] last=Optional[r2] 2023-03-08 07:53:51,682 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8a36d79128d34588858cdbd3758340af}] 2023-03-08 07:53:51,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8a36d79128d34588858cdbd3758340af for inclusion in 38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:51,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 07:53:51,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:53:51,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:51,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8a36d79128d34588858cdbd3758340af to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__usj00v6ma1mvppdpmulmb10ta2uebv2brolft779p2eic7i0535617gml6fft6k4/family1/8a36d79128d34588858cdbd3758340af 2023-03-08 07:53:51,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__usj00v6ma1mvppdpmulmb10ta2uebv2brolft779p2eic7i0535617gml6fft6k4/family1/8a36d79128d34588858cdbd3758340af as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_ 2023-03-08 07:53:51,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__usj00v6ma1mvppdpmulmb10ta2uebv2brolft779p2eic7i0535617gml6fft6k4/family1/8a36d79128d34588858cdbd3758340af into 38efdc462c389fa50e647861e0aace4c/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_ - updating store file list. 2023-03-08 07:53:51,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_ into 38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:51,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__usj00v6ma1mvppdpmulmb10ta2uebv2brolft779p2eic7i0535617gml6fft6k4/family1/8a36d79128d34588858cdbd3758340af into 38efdc462c389fa50e647861e0aace4c/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_) 2023-03-08 07:53:51,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__usj00v6ma1mvppdpmulmb10ta2uebv2brolft779p2eic7i0535617gml6fft6k4/family1/8a36d79128d34588858cdbd3758340af 2023-03-08 07:53:51,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 07:53:51,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 07:53:51,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 38efdc462c389fa50e647861e0aace4c/family1 is initiating minor compaction (all files) 2023-03-08 07:53:51,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@57ee4952; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:53:51,703 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 38efdc462c389fa50e647861e0aace4c/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:51,704 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/e2c16c94ac094a0fb6d80980812c898a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/26d22426baf84eb9b47dffc610db496b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp, totalSize=358.6 K 2023-03-08 07:53:51,705 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting e2c16c94ac094a0fb6d80980812c898a, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839991 2023-03-08 07:53:51,705 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 26d22426baf84eb9b47dffc610db496b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239991 2023-03-08 07:53:51,706 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839991 2023-03-08 07:53:51,706 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:51,708 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:53:51,710 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300d3bd2f0009c495cb602ef03dc0399c8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300d3bd2f0009c495cb602ef03dc0399c8 2023-03-08 07:53:51,711 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201585b9f1c105c4753a19e3bf507a78754 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201585b9f1c105c4753a19e3bf507a78754 2023-03-08 07:53:51,713 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057ff8708ddf6f4236a3959a58774bd97e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057ff8708ddf6f4236a3959a58774bd97e 2023-03-08 07:53:51,713 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-08 07:53:51,714 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 07:53:51,714 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:51,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:51,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:53:51,716 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 07:53:51,718 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6000 sec 2023-03-08 07:53:51,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/d6a4ef896044462383852adc29ae8b0e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5fcb5dcc907b4e51ac485a54976989d8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/cc828dc770c64a02ba19d41f7ebc6703, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a1829bb82f864f5fb3b10e88659b09b3, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dadf754747ec4bc18649ec3fd6305da1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5f0b8a4cc77c484cae3d164e165865ee, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/3e4f8f35a1c547158d9a319e6a5c3cb0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/0ebf25f719cc439ea59f6fc6626902cb, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a259443066e147689031a2d4aa2f37e0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_] to archive 2023-03-08 07:53:51,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:53:51,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/d6a4ef896044462383852adc29ae8b0e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/d6a4ef896044462383852adc29ae8b0e 2023-03-08 07:53:51,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5fcb5dcc907b4e51ac485a54976989d8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5fcb5dcc907b4e51ac485a54976989d8 2023-03-08 07:53:51,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/cc828dc770c64a02ba19d41f7ebc6703 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/cc828dc770c64a02ba19d41f7ebc6703 2023-03-08 07:53:51,747 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a1829bb82f864f5fb3b10e88659b09b3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a1829bb82f864f5fb3b10e88659b09b3 2023-03-08 07:53:51,749 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dadf754747ec4bc18649ec3fd6305da1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dadf754747ec4bc18649ec3fd6305da1 2023-03-08 07:53:51,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5f0b8a4cc77c484cae3d164e165865ee to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5f0b8a4cc77c484cae3d164e165865ee 2023-03-08 07:53:51,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/3e4f8f35a1c547158d9a319e6a5c3cb0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/3e4f8f35a1c547158d9a319e6a5c3cb0 2023-03-08 07:53:51,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/0ebf25f719cc439ea59f6fc6626902cb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/0ebf25f719cc439ea59f6fc6626902cb 2023-03-08 07:53:51,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a259443066e147689031a2d4aa2f37e0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/a259443066e147689031a2d4aa2f37e0 2023-03-08 07:53:51,763 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/5106ccdc82bd44e2840b1c3a94433450_SeqId_44_ 2023-03-08 07:53:51,766 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 38efdc462c389fa50e647861e0aace4c#family1#compaction#11 average throughput is 4.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:53:51,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 07:53:51,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:52,320 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 07:53:52,323 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 07:53:52,323 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 07:53:52,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-08 07:53:52,340 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, REOPEN/MOVE}] 2023-03-08 07:53:52,341 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, REOPEN/MOVE 2023-03-08 07:53:52,341 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=38efdc462c389fa50e647861e0aace4c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:52,342 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262032341"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262032341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262032341"}]},"ts":"1678262032341"} 2023-03-08 07:53:52,344 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 38efdc462c389fa50e647861e0aace4c, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:53:52,502 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:52,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 38efdc462c389fa50e647861e0aace4c, disabling compactions & flushes 2023-03-08 07:53:52,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/.tmp/family1/90079c0455db45798796e71547969610 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/90079c0455db45798796e71547969610 2023-03-08 07:53:53,009 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 38efdc462c389fa50e647861e0aace4c/family1 of 38efdc462c389fa50e647861e0aace4c into 90079c0455db45798796e71547969610(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:53:53,009 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:53,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. after waiting 0 ms 2023-03-08 07:53:53,009 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678262031703; duration=1sec 2023-03-08 07:53:53,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,009 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:53:53,012 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/e2c16c94ac094a0fb6d80980812c898a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/26d22426baf84eb9b47dffc610db496b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_] to archive 2023-03-08 07:53:53,013 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:53:53,015 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/e2c16c94ac094a0fb6d80980812c898a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/e2c16c94ac094a0fb6d80980812c898a 2023-03-08 07:53:53,017 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/26d22426baf84eb9b47dffc610db496b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/26d22426baf84eb9b47dffc610db496b 2023-03-08 07:53:53,019 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/dc0894a4387a438eb91c0765a2e89b9d_SeqId_47_ 2023-03-08 07:53:53,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 07:53:53,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:53,026 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 38efdc462c389fa50e647861e0aace4c to self. 2023-03-08 07:53:53,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,029 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=38efdc462c389fa50e647861e0aace4c, regionState=CLOSED 2023-03-08 07:53:53,029 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262033029"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262033029"}]},"ts":"1678262033029"} 2023-03-08 07:53:53,034 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-08 07:53:53,034 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 38efdc462c389fa50e647861e0aace4c, server=jenkins-hbase4.apache.org,33669,1678262016084 in 687 msec 2023-03-08 07:53:53,034 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=true 2023-03-08 07:53:53,185 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=38efdc462c389fa50e647861e0aace4c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:53,185 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262033185"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262033185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262033185"}]},"ts":"1678262033185"} 2023-03-08 07:53:53,188 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 38efdc462c389fa50e647861e0aace4c, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:53:53,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 38efdc462c389fa50e647861e0aace4c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:53:53,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:53,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,346 INFO [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,347 DEBUG [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:53,347 DEBUG [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:53,348 INFO [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38efdc462c389fa50e647861e0aace4c columnFamilyName family1 2023-03-08 07:53:53,355 DEBUG [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/90079c0455db45798796e71547969610 2023-03-08 07:53:53,356 INFO [StoreOpener-38efdc462c389fa50e647861e0aace4c-1] regionserver.HStore(310): Store=38efdc462c389fa50e647861e0aace4c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:53,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:53:53,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 38efdc462c389fa50e647861e0aace4c; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60211330, jitterRate=-0.1027812659740448}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:53:53,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:53,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., pid=17, masterSystemTime=1678262033340 2023-03-08 07:53:53,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:53:53,366 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=38efdc462c389fa50e647861e0aace4c, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:53,366 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678262033366"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262033366"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262033366"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262033366"}]},"ts":"1678262033366"} 2023-03-08 07:53:53,370 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-08 07:53:53,370 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 38efdc462c389fa50e647861e0aace4c, server=jenkins-hbase4.apache.org,33669,1678262016084 in 180 msec 2023-03-08 07:53:53,371 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-08 07:53:53,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=38efdc462c389fa50e647861e0aace4c, REOPEN/MOVE in 1.0300 sec 2023-03-08 07:53:53,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-08 07:53:53,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0470 sec 2023-03-08 07:53:53,379 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4780 sec 2023-03-08 07:53:53,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-08 07:53:53,391 INFO [Listener at localhost/42999] compactions.TestMobCompactor(278): alter status finished 2023-03-08 07:53:53,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 07:53:53,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 07:53:53,397 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 07:53:53,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:53:53,398 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262633398 2023-03-08 07:53:53,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:53:53,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 07:53:53,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:53,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x165476fd to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:53:53,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a76ab6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:53:53,435 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:53,435 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:53,435 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:53,435 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:53,435 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:54,277 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285a8d50e556fc49acae6c7d2133286cde to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285a8d50e556fc49acae6c7d2133286cde 2023-03-08 07:53:54,278 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:54,292 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:53:54,293 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53314, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:53:54,302 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:53:54,304 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46694, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:53:54,315 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/08bf0e51811640e194f1975f777e9aec first=Optional[r05] last=Optional[r8] 2023-03-08 07:53:54,316 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/08bf0e51811640e194f1975f777e9aec}] 2023-03-08 07:53:54,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/08bf0e51811640e194f1975f777e9aec for inclusion in 38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:54,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 07:53:54,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:53:54,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:53:54,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/08bf0e51811640e194f1975f777e9aec to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hib9b3d6n1fbiefc8ghak0064gjp4k9ojfsg2mv930f70sps5a3es7uoe1731eep/family1/08bf0e51811640e194f1975f777e9aec 2023-03-08 07:53:54,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hib9b3d6n1fbiefc8ghak0064gjp4k9ojfsg2mv930f70sps5a3es7uoe1731eep/family1/08bf0e51811640e194f1975f777e9aec as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/b7094160f13449e2aa6313b91912acc5_SeqId_53_ 2023-03-08 07:53:54,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hib9b3d6n1fbiefc8ghak0064gjp4k9ojfsg2mv930f70sps5a3es7uoe1731eep/family1/08bf0e51811640e194f1975f777e9aec into 38efdc462c389fa50e647861e0aace4c/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/b7094160f13449e2aa6313b91912acc5_SeqId_53_ - updating store file list. 2023-03-08 07:53:54,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/b7094160f13449e2aa6313b91912acc5_SeqId_53_ into 38efdc462c389fa50e647861e0aace4c/family1 2023-03-08 07:53:54,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hib9b3d6n1fbiefc8ghak0064gjp4k9ojfsg2mv930f70sps5a3es7uoe1731eep/family1/08bf0e51811640e194f1975f777e9aec into 38efdc462c389fa50e647861e0aace4c/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/family1/b7094160f13449e2aa6313b91912acc5_SeqId_53_) 2023-03-08 07:53:54,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hib9b3d6n1fbiefc8ghak0064gjp4k9ojfsg2mv930f70sps5a3es7uoe1731eep/family1/08bf0e51811640e194f1975f777e9aec 2023-03-08 07:53:54,340 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:54,341 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:53:54,343 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120b938e565ed5e490788a7395caf35fddc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120b938e565ed5e490788a7395caf35fddc 2023-03-08 07:53:54,344 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289ff923bac42f4ded8330bf681995a124 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289ff923bac42f4ded8330bf681995a124 2023-03-08 07:53:54,344 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-08 07:53:54,344 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 07:53:54,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:54,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:54,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 07:53:54,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 07:53:54,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:53:54,346 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 07:53:54,348 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 953 msec 2023-03-08 07:53:54,424 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 211) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39418 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39248 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39224 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39310 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x71dc5e96-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52387@0x59d158ad sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1216092301.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39424 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52387@0x165476fd-SendThread(127.0.0.1:52387) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:33669-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39240 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fd62c60-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52387@0x165476fd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39178 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39344 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fd62c60-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x71dc5e96-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39336 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52387@0x59d158ad-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39292 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39320 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_119793848_15 at /127.0.0.1:39234 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fd62c60-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39280 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39186 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52387@0x59d158ad-SendThread(127.0.0.1:52387) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52387@0x165476fd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1216092301.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39218 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39144 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_879761085_15 at /127.0.0.1:39356 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=598 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=351 (was 401), ProcessCount=170 (was 170), AvailableMemoryMB=7463 (was 7802) 2023-03-08 07:53:54,436 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=170, AvailableMemoryMB=7462 2023-03-08 07:53:54,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 07:53:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:54,441 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:53:54,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-08 07:53:54,442 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:53:54,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 07:53:54,444 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:54,445 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1 empty. 2023-03-08 07:53:54,445 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:54,445 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-08 07:53:54,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 07:53:54,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 07:53:54,855 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 07:53:54,856 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => fbe4dd1fee45d9edc4c4fe869fc49ac1, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:53:55,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 07:53:55,263 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:55,264 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing fbe4dd1fee45d9edc4c4fe869fc49ac1, disabling compactions & flushes 2023-03-08 07:53:55,264 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,264 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,264 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. after waiting 0 ms 2023-03-08 07:53:55,264 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,264 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,264 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:53:55,268 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:53:55,269 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678262035269"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262035269"}]},"ts":"1678262035269"} 2023-03-08 07:53:55,272 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:53:55,273 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:53:55,274 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262035273"}]},"ts":"1678262035273"} 2023-03-08 07:53:55,276 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 07:53:55,285 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fbe4dd1fee45d9edc4c4fe869fc49ac1, ASSIGN}] 2023-03-08 07:53:55,287 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fbe4dd1fee45d9edc4c4fe869fc49ac1, ASSIGN 2023-03-08 07:53:55,289 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fbe4dd1fee45d9edc4c4fe869fc49ac1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:53:55,440 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fbe4dd1fee45d9edc4c4fe869fc49ac1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:55,441 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678262035440"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262035440"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262035440"}]},"ts":"1678262035440"} 2023-03-08 07:53:55,443 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure fbe4dd1fee45d9edc4c4fe869fc49ac1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:53:55,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 07:53:55,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fbe4dd1fee45d9edc4c4fe869fc49ac1, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:53:55,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:53:55,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,601 INFO [StoreOpener-fbe4dd1fee45d9edc4c4fe869fc49ac1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,602 DEBUG [StoreOpener-fbe4dd1fee45d9edc4c4fe869fc49ac1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:53:55,602 DEBUG [StoreOpener-fbe4dd1fee45d9edc4c4fe869fc49ac1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:53:55,602 INFO [StoreOpener-fbe4dd1fee45d9edc4c4fe869fc49ac1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fbe4dd1fee45d9edc4c4fe869fc49ac1 columnFamilyName family1 2023-03-08 07:53:55,604 INFO [StoreOpener-fbe4dd1fee45d9edc4c4fe869fc49ac1-1] regionserver.HStore(310): Store=fbe4dd1fee45d9edc4c4fe869fc49ac1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:53:55,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:53:55,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:53:55,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fbe4dd1fee45d9edc4c4fe869fc49ac1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71659219, jitterRate=0.06780557334423065}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:53:55,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:53:55,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1., pid=21, masterSystemTime=1678262035595 2023-03-08 07:53:55,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:55,614 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fbe4dd1fee45d9edc4c4fe869fc49ac1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:55,614 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678262035614"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262035614"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262035614"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262035614"}]},"ts":"1678262035614"} 2023-03-08 07:53:55,617 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-08 07:53:55,617 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure fbe4dd1fee45d9edc4c4fe869fc49ac1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 173 msec 2023-03-08 07:53:55,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-08 07:53:55,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fbe4dd1fee45d9edc4c4fe869fc49ac1, ASSIGN in 332 msec 2023-03-08 07:53:55,620 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:53:55,620 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262035620"}]},"ts":"1678262035620"} 2023-03-08 07:53:55,622 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 07:53:55,624 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:53:55,625 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1850 sec 2023-03-08 07:53:55,773 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:56,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 07:53:56,550 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-08 07:53:56,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:56,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,581 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:56,581 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:56,581 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:53:56,581 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:53:56,582 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,582 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,584 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:56,584 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:56,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:56,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:56,585 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:56,585 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:56,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:56,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,585 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,586 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:56,586 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:53:56,586 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:56,586 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:53:56,586 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:56,586 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:56,586 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:56,586 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:53:56,587 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:53:57,012 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bdfe4c4951154adc80b248ab840f37d3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130bdfe4c4951154adc80b248ab840f37d3 2023-03-08 07:53:57,417 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/ba7d06a745c04ce4a20c5a7d6ed533bf 2023-03-08 07:53:57,425 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/ba7d06a745c04ce4a20c5a7d6ed533bf as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/ba7d06a745c04ce4a20c5a7d6ed533bf 2023-03-08 07:53:57,430 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/ba7d06a745c04ce4a20c5a7d6ed533bf, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 07:53:57,432 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 845ms, sequenceid=5, compaction requested=false 2023-03-08 07:53:57,432 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:53:57,432 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:57,432 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:57,432 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:57,433 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,433 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:53:57,433 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:53:57,434 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,434 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:57,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:57,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,435 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:57,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:57,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:57,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:57,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:53:57,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39f760b2[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:57,436 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:53:57,436 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,438 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:57,438 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,439 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:57,440 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:53:57,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:57,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:57,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:57,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:53:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:57,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:53:57,443 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:53:57,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:57,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:57,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,443 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:57,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:57,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:57,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:57,445 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,446 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,449 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:57,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:57,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:57,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:57,449 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:57,450 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:57,450 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:57,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:57,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:57,550 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:57,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:57,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:57,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:57,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,561 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:57,561 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:57,562 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:53:57,562 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:53:57,562 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,562 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,565 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,565 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:57,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:57,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:57,565 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:57,565 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:57,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:57,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:53:57,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:53:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:53:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:57,568 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:57,568 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:57,568 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:53:57,568 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:53:57,987 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c949d79f83764e5190c7aa27c319d4de to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128c949d79f83764e5190c7aa27c319d4de 2023-03-08 07:53:58,393 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/fb14c6fe85f04e60b159e08424e0650e 2023-03-08 07:53:58,400 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/fb14c6fe85f04e60b159e08424e0650e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/fb14c6fe85f04e60b159e08424e0650e 2023-03-08 07:53:58,408 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/fb14c6fe85f04e60b159e08424e0650e, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:53:58,410 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 842ms, sequenceid=9, compaction requested=false 2023-03-08 07:53:58,410 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:53:58,410 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:58,410 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:58,410 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:58,410 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,410 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:53:58,410 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:53:58,412 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,412 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:58,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:58,413 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:58,413 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:58,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:58,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:58,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:53:58,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7840f55d[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:58,415 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:53:58,415 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,416 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,416 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:53:58,416 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:53:58,416 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:58,416 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,416 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:58,418 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,418 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:53:58,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:58,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:58,418 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:58,418 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:53:58,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:58,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:58,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:58,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:58,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:53:58,422 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:53:58,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:58,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:58,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,422 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:58,425 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,425 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:58,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,425 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:58,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:58,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:58,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:58,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:58,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,426 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:58,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,432 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,443 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:58,443 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:58,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:58,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:58,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:58,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:58,443 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:58,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,444 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,544 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:58,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:58,551 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:58,552 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:53:58,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:58,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,554 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:58,554 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:58,555 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:53:58,555 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:53:58,555 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,555 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,609 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,609 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:58,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:58,610 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:58,610 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:53:58,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:58,611 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:53:58,611 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:58,612 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:53:58,612 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:58,612 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:58,612 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:58,612 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:53:58,612 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:53:59,028 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010496ba853bf3437fad82b3556dccce71 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010496ba853bf3437fad82b3556dccce71 2023-03-08 07:53:59,434 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/456c6cd5dec94fc1b5861ca66d7619f2 2023-03-08 07:53:59,443 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/456c6cd5dec94fc1b5861ca66d7619f2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/456c6cd5dec94fc1b5861ca66d7619f2 2023-03-08 07:53:59,450 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/456c6cd5dec94fc1b5861ca66d7619f2, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 07:53:59,451 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 839ms, sequenceid=13, compaction requested=true 2023-03-08 07:53:59,451 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:53:59,451 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:59,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:53:59,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:53:59,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:53:59,452 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:53:59,486 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,486 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:59,487 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,487 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:53:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:59,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:59,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:53:59,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b2e2314[Count = 0] remaining members to acquire global barrier 2023-03-08 07:53:59,489 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:53:59,489 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,525 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,525 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:53:59,525 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,525 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:53:59,525 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:53:59,525 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:53:59,548 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:53:59,548 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,548 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:59,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:59,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:59,548 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:53:59,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:53:59,551 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:53:59,551 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:53:59,551 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:53:59,552 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,552 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:53:59,556 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,556 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:53:59,556 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,557 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:53:59,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:53:59,563 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:53:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,567 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:59,567 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:53:59,567 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:53:59,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:59,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:59,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:53:59,567 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:53:59,568 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:53:59,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:53:59,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:53:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:53:59,668 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:53:59,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:53:59,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:53:59,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:53:59,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,677 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:53:59,677 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:59,678 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:53:59,678 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:53:59,678 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,678 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,680 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,680 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:53:59,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:53:59,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:53:59,680 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:53:59,680 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:53:59,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:53:59,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,681 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,681 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:53:59,681 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:53:59,681 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:53:59,681 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:53:59,682 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:53:59,682 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:53:59,682 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:53:59,682 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:53:59,682 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:53:59,682 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:00,100 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120563b8bedfb4cf482ebb7be8f4ba8ce139 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120563b8bedfb4cf482ebb7be8f4ba8ce139 2023-03-08 07:54:00,513 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/19ea44caeea947a897274d0e9200b3dd 2023-03-08 07:54:00,521 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/19ea44caeea947a897274d0e9200b3dd as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/19ea44caeea947a897274d0e9200b3dd 2023-03-08 07:54:00,529 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/19ea44caeea947a897274d0e9200b3dd, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 07:54:00,530 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 848ms, sequenceid=17, compaction requested=true 2023-03-08 07:54:00,530 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:00,530 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:00,531 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:00,532 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,532 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:00,533 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,533 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:00,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:00,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35b3b62c[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:00,535 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:00,535 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,538 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,538 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:00,538 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:00,538 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:00,538 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,538 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:00,540 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:00,540 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:00,540 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,540 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:00,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:00,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:00,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:00,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:00,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:00,544 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:00,544 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:00,544 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:00,544 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,544 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:00,547 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,547 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:00,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,547 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:00,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:00,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:00,547 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,549 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:00,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,552 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:00,552 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:00,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:00,553 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,553 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:00,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:00,556 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:00,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:00,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:54:00,657 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:00,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:00,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:00,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:00,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,669 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:00,669 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:00,670 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:54:00,670 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:00,672 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,672 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,678 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,678 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:00,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:00,679 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:00,679 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,679 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,680 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,682 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:54:00,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:00,682 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:54:00,682 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:00,683 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:54:00,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:00,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:00,684 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:00,684 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:54:00,684 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:00,713 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228a183318f43894aaeab007a72818bcb08 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228a183318f43894aaeab007a72818bcb08 2023-03-08 07:54:01,124 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/f7412d8f314b4d3eba8b3e007197dd0a 2023-03-08 07:54:01,130 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/f7412d8f314b4d3eba8b3e007197dd0a as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/f7412d8f314b4d3eba8b3e007197dd0a 2023-03-08 07:54:01,135 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/f7412d8f314b4d3eba8b3e007197dd0a, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 07:54:01,136 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 452ms, sequenceid=21, compaction requested=true 2023-03-08 07:54:01,136 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:01,136 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:01,136 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:01,136 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:01,136 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,136 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:01,136 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:01,138 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:01,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:01,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:01,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:01,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3743a2d8[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:01,140 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:01,140 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,141 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:01,141 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,141 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:01,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:01,142 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:01,143 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:01,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:01,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:01,145 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:01,145 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:01,145 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:01,145 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,145 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:01,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:01,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,154 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:01,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,155 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,157 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,157 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:01,157 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:01,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:01,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:01,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:01,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:01,158 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:54:01,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:01,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:01,159 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:01,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,159 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:01,259 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:01,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:01,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:01,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:01,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,267 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:01,267 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:01,267 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:54:01,268 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:01,268 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,268 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,270 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,270 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:01,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:01,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:01,270 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:01,270 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:01,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:01,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:54:01,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:01,272 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:54:01,272 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:01,272 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:54:01,272 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:01,272 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:01,273 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:01,273 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:54:01,273 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:01,688 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d5075456ec1d4087bdc52944f69a36ba to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d5075456ec1d4087bdc52944f69a36ba 2023-03-08 07:54:02,094 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5 2023-03-08 07:54:02,101 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5 2023-03-08 07:54:02,106 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 07:54:02,107 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 834ms, sequenceid=25, compaction requested=true 2023-03-08 07:54:02,107 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:02,107 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:02,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:02,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:02,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:02,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:02,109 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,109 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:02,109 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,109 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48d64370[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:02,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:02,111 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,112 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:02,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:02,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,113 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:02,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:02,114 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:02,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:02,117 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:02,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:02,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:02,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,117 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:02,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:02,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:02,119 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:02,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,120 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:02,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,122 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:02,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,123 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:02,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:02,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:02,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:02,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:02,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:02,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:02,124 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:02,124 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,124 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,124 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:02,124 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:02,224 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:02,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:02,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:02,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:02,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,231 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:02,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:02,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:54:02,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:02,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,233 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,234 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,234 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:02,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:02,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:02,234 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:02,235 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,235 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:54:02,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:54:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:54:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:02,236 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:02,236 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:54:02,236 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:02,652 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101869e262e6f224eb5b0f3726f3ebfe874 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101869e262e6f224eb5b0f3726f3ebfe874 2023-03-08 07:54:03,058 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/d2f2b11e3e4a40efb6dd413277699881 2023-03-08 07:54:03,064 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/d2f2b11e3e4a40efb6dd413277699881 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d2f2b11e3e4a40efb6dd413277699881 2023-03-08 07:54:03,068 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d2f2b11e3e4a40efb6dd413277699881, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 07:54:03,069 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 833ms, sequenceid=29, compaction requested=true 2023-03-08 07:54:03,069 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:03,069 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:03,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:03,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:03,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:03,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:03,071 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,071 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:03,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:03,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:03,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:03,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:03,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:03,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:03,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12baeffd[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:03,073 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:03,073 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,074 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,074 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:03,074 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:03,074 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:03,074 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,074 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:03,075 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:03,075 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,075 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:03,076 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:03,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:03,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:03,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:03,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:03,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:03,078 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:03,078 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:03,078 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:03,078 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,078 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:03,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:03,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:03,081 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:03,082 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:03,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,084 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,084 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,084 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:03,084 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:03,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:03,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:03,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:03,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:54:03,084 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:03,084 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:03,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:03,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:03,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,085 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:03,086 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:03,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,186 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:03,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:03,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:03,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:03,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,194 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:03,195 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:03,195 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:54:03,195 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:03,195 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,195 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,197 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:03,197 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:03,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:03,197 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:03,198 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,198 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,198 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:54:03,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:03,199 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:03,199 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:54:03,200 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:03,615 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037f95a92a423c4cafa1b895641d12a498 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601037f95a92a423c4cafa1b895641d12a498 2023-03-08 07:54:04,021 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/d4fd0dd74213412ab20316410c4f64e7 2023-03-08 07:54:04,027 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/d4fd0dd74213412ab20316410c4f64e7 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d4fd0dd74213412ab20316410c4f64e7 2023-03-08 07:54:04,032 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d4fd0dd74213412ab20316410c4f64e7, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 07:54:04,033 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 833ms, sequenceid=33, compaction requested=true 2023-03-08 07:54:04,033 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:04,033 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:04,033 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:04,033 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:04,033 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,033 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:04,033 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:04,035 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,035 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:04,035 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,035 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:04,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:04,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a902950[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:04,037 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:04,037 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,039 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,039 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:04,039 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:04,039 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:04,039 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,039 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:04,043 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,043 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:04,043 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:04,044 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:04,046 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:04,046 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:04,046 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:04,046 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,046 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:04,048 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,048 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,048 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:04,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:04,048 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:04,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:04,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,049 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:04,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:04,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,053 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:04,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:04,054 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:04,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:04,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:04,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:04,054 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:04,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:04,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:04,154 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:04,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:04,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:04,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:04,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,163 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:04,163 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:04,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:54:04,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:04,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,166 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,166 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:04,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:04,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:04,166 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:04,166 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:04,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:54:04,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:04,167 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:54:04,167 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:04,167 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:54:04,168 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:04,168 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:04,168 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:04,168 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:54:04,168 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:04,583 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090758a9af48c49346a193ee757c8b160910 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015090758a9af48c49346a193ee757c8b160910 2023-03-08 07:54:04,989 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/120a6e9f6c154a59ac6fa5901221936a 2023-03-08 07:54:04,995 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/120a6e9f6c154a59ac6fa5901221936a as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/120a6e9f6c154a59ac6fa5901221936a 2023-03-08 07:54:05,000 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/120a6e9f6c154a59ac6fa5901221936a, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 07:54:05,001 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 833ms, sequenceid=37, compaction requested=true 2023-03-08 07:54:05,001 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:05,001 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:05,001 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:05,001 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:05,001 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,001 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:05,001 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:05,003 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,003 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:05,003 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,003 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:05,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:05,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@106f09b8[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:05,005 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:05,005 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,006 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:05,006 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,006 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:05,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:05,008 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:05,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:05,010 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:05,010 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:05,010 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:05,010 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,010 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:05,013 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,013 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:05,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:05,013 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:05,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:05,014 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:05,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:05,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:05,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,015 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:05,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,017 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,017 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:05,017 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:05,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:05,017 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:05,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:05,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:05,017 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:05,017 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:05,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:05,018 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,018 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:05,018 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,019 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:05,019 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,019 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,119 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:05,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:05,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:05,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:05,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,126 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:05,126 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:05,127 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:54:05,127 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:05,127 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,127 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,129 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:05,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:05,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:05,129 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,129 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:05,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:05,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:54:05,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,130 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:54:05,130 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:05,131 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:54:05,132 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:05,132 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:05,132 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:05,132 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. started... 2023-03-08 07:54:05,132 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fbe4dd1fee45d9edc4c4fe869fc49ac1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:05,547 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112078ea3db9e4ed41bcb4c10abbc5bf870a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112078ea3db9e4ed41bcb4c10abbc5bf870a 2023-03-08 07:54:05,953 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/6409748420744c3d82a08bad2295f4ed 2023-03-08 07:54:05,958 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/6409748420744c3d82a08bad2295f4ed as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/6409748420744c3d82a08bad2295f4ed 2023-03-08 07:54:05,964 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/6409748420744c3d82a08bad2295f4ed, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 07:54:05,965 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fbe4dd1fee45d9edc4c4fe869fc49ac1 in 833ms, sequenceid=41, compaction requested=true 2023-03-08 07:54:05,965 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:05,965 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:05,965 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:05,965 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:05,965 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,965 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:54:05,965 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:54:05,967 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,967 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:05,967 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,967 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ad80506[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:05,968 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:05,968 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,970 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,971 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:05,971 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:54:05,971 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,971 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:05,971 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:05,972 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:05,972 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:54:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:05,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:05,975 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:54:05,975 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:05,975 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:05,975 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,975 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:05,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:05,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,976 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:05,977 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,978 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:05,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:05,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:05,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:54:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:05,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 07:54:05,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:05,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:05,980 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:05,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:05,981 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:05,982 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:05,982 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:05,982 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:05,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:05,982 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:06,082 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:06,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:06,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-08 07:54:06,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:06,087 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:06,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:06,088 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262646088 2023-03-08 07:54:06,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:54:06,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 07:54:06,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:06,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x2205f02c to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:06,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6de70ad0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:06,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:06,927 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010307f0bdc314da4b47bba8ea05881974fc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010307f0bdc314da4b47bba8ea05881974fc 2023-03-08 07:54:06,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:06,941 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:54:06,942 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38032, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:54:06,949 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:54:06,950 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50912, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:54:06,961 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8cd30c3fa0f7448da41d4be1c6a67a03 first=Optional[r5] last=Optional[r6] 2023-03-08 07:54:06,962 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8cd30c3fa0f7448da41d4be1c6a67a03}] 2023-03-08 07:54:06,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8cd30c3fa0f7448da41d4be1c6a67a03 for inclusion in fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:54:06,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 07:54:06,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:06,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:06,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/8cd30c3fa0f7448da41d4be1c6a67a03 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5n5eskcujte2ubtg2087m2svhe0dr20f2dt4tovu9ukie4r5uitb9ellsuis087n/family1/8cd30c3fa0f7448da41d4be1c6a67a03 2023-03-08 07:54:06,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5n5eskcujte2ubtg2087m2svhe0dr20f2dt4tovu9ukie4r5uitb9ellsuis087n/family1/8cd30c3fa0f7448da41d4be1c6a67a03 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_ 2023-03-08 07:54:06,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5n5eskcujte2ubtg2087m2svhe0dr20f2dt4tovu9ukie4r5uitb9ellsuis087n/family1/8cd30c3fa0f7448da41d4be1c6a67a03 into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_ - updating store file list. 2023-03-08 07:54:06,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_ into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:54:06,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5n5eskcujte2ubtg2087m2svhe0dr20f2dt4tovu9ukie4r5uitb9ellsuis087n/family1/8cd30c3fa0f7448da41d4be1c6a67a03 into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_) 2023-03-08 07:54:06,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5n5eskcujte2ubtg2087m2svhe0dr20f2dt4tovu9ukie4r5uitb9ellsuis087n/family1/8cd30c3fa0f7448da41d4be1c6a67a03 2023-03-08 07:54:06,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:54:06,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:54:06,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 is initiating minor compaction 2023-03-08 07:54:06,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6fc30e35; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:06,982 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 in testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:06,982 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/fb14c6fe85f04e60b159e08424e0650e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/456c6cd5dec94fc1b5861ca66d7619f2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/19ea44caeea947a897274d0e9200b3dd, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/f7412d8f314b4d3eba8b3e007197dd0a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d2f2b11e3e4a40efb6dd413277699881, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d4fd0dd74213412ab20316410c4f64e7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/120a6e9f6c154a59ac6fa5901221936a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/6409748420744c3d82a08bad2295f4ed, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp, totalSize=49.7 K 2023-03-08 07:54:06,982 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting fb14c6fe85f04e60b159e08424e0650e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:54:06,982 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 456c6cd5dec94fc1b5861ca66d7619f2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:54:06,982 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 19ea44caeea947a897274d0e9200b3dd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting f7412d8f314b4d3eba8b3e007197dd0a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting cfbe1fa512e546c886c4b5bd9c9e3bd5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting d2f2b11e3e4a40efb6dd413277699881, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting d4fd0dd74213412ab20316410c4f64e7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 120a6e9f6c154a59ac6fa5901221936a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 6409748420744c3d82a08bad2295f4ed, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:54:06,983 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 64100875dd7f4679a8e23867c6bc07f4_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:54:06,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:06,985 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:06,986 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101869e262e6f224eb5b0f3726f3ebfe874 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101869e262e6f224eb5b0f3726f3ebfe874 2023-03-08 07:54:06,988 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601037f95a92a423c4cafa1b895641d12a498 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601037f95a92a423c4cafa1b895641d12a498 2023-03-08 07:54:06,988 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 07:54:06,988 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 07:54:06,988 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:06,988 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:07,020 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): fbe4dd1fee45d9edc4c4fe869fc49ac1#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:54:07,442 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/4583bc717488475a95b54dde0f2357f2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4583bc717488475a95b54dde0f2357f2 2023-03-08 07:54:07,448 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c9ecb54c2bbf4d519746566ca0446f81 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c9ecb54c2bbf4d519746566ca0446f81 2023-03-08 07:54:07,449 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:07,449 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 of fbe4dd1fee45d9edc4c4fe869fc49ac1 into 4583bc717488475a95b54dde0f2357f2(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:54:07,449 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:07,450 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678262046982; duration=0sec 2023-03-08 07:54:07,450 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:07,475 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/80b31f07652f4c2e96bdbf8eab048d22 first=Optional[r1] last=Optional[r4] 2023-03-08 07:54:07,476 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/80b31f07652f4c2e96bdbf8eab048d22}] 2023-03-08 07:54:07,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/80b31f07652f4c2e96bdbf8eab048d22 for inclusion in fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:54:07,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 07:54:07,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:07,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:07,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/80b31f07652f4c2e96bdbf8eab048d22 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2ju0kbl3d2v7u7fr1rnh9ib0hg1hae8u0o4r2jdj7ag4pe12ih7fib68i36olkiq/family1/80b31f07652f4c2e96bdbf8eab048d22 2023-03-08 07:54:07,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2ju0kbl3d2v7u7fr1rnh9ib0hg1hae8u0o4r2jdj7ag4pe12ih7fib68i36olkiq/family1/80b31f07652f4c2e96bdbf8eab048d22 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_ 2023-03-08 07:54:07,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2ju0kbl3d2v7u7fr1rnh9ib0hg1hae8u0o4r2jdj7ag4pe12ih7fib68i36olkiq/family1/80b31f07652f4c2e96bdbf8eab048d22 into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_ - updating store file list. 2023-03-08 07:54:07,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_ into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:54:07,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2ju0kbl3d2v7u7fr1rnh9ib0hg1hae8u0o4r2jdj7ag4pe12ih7fib68i36olkiq/family1/80b31f07652f4c2e96bdbf8eab048d22 into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_) 2023-03-08 07:54:07,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2ju0kbl3d2v7u7fr1rnh9ib0hg1hae8u0o4r2jdj7ag4pe12ih7fib68i36olkiq/family1/80b31f07652f4c2e96bdbf8eab048d22 2023-03-08 07:54:07,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 07:54:07,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 07:54:07,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 is initiating minor compaction (all files) 2023-03-08 07:54:07,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@41249676; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:07,492 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 in testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:54:07,492 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/ba7d06a745c04ce4a20c5a7d6ed533bf, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4583bc717488475a95b54dde0f2357f2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp, totalSize=178.1 K 2023-03-08 07:54:07,493 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ba7d06a745c04ce4a20c5a7d6ed533bf, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839991 2023-03-08 07:54:07,494 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 4583bc717488475a95b54dde0f2357f2, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239991 2023-03-08 07:54:07,494 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting bce47919ac1c43f492e917599458126c_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239991 2023-03-08 07:54:07,495 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:07,495 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:07,497 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010496ba853bf3437fad82b3556dccce71 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010496ba853bf3437fad82b3556dccce71 2023-03-08 07:54:07,499 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120563b8bedfb4cf482ebb7be8f4ba8ce139 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120563b8bedfb4cf482ebb7be8f4ba8ce139 2023-03-08 07:54:07,500 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228a183318f43894aaeab007a72818bcb08 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228a183318f43894aaeab007a72818bcb08 2023-03-08 07:54:07,501 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d5075456ec1d4087bdc52944f69a36ba to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d5075456ec1d4087bdc52944f69a36ba 2023-03-08 07:54:07,501 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 07:54:07,501 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 07:54:07,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:07,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:07,516 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): fbe4dd1fee45d9edc4c4fe869fc49ac1#family1#compaction#23 average throughput is 11.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:54:07,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/fb14c6fe85f04e60b159e08424e0650e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/456c6cd5dec94fc1b5861ca66d7619f2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/19ea44caeea947a897274d0e9200b3dd, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/f7412d8f314b4d3eba8b3e007197dd0a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d2f2b11e3e4a40efb6dd413277699881, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d4fd0dd74213412ab20316410c4f64e7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/120a6e9f6c154a59ac6fa5901221936a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/6409748420744c3d82a08bad2295f4ed, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_] to archive 2023-03-08 07:54:07,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:07,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/fb14c6fe85f04e60b159e08424e0650e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/fb14c6fe85f04e60b159e08424e0650e 2023-03-08 07:54:07,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/456c6cd5dec94fc1b5861ca66d7619f2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/456c6cd5dec94fc1b5861ca66d7619f2 2023-03-08 07:54:07,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/19ea44caeea947a897274d0e9200b3dd to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/19ea44caeea947a897274d0e9200b3dd 2023-03-08 07:54:07,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/f7412d8f314b4d3eba8b3e007197dd0a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/f7412d8f314b4d3eba8b3e007197dd0a 2023-03-08 07:54:07,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/cfbe1fa512e546c886c4b5bd9c9e3bd5 2023-03-08 07:54:07,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d2f2b11e3e4a40efb6dd413277699881 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d2f2b11e3e4a40efb6dd413277699881 2023-03-08 07:54:07,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d4fd0dd74213412ab20316410c4f64e7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/d4fd0dd74213412ab20316410c4f64e7 2023-03-08 07:54:07,557 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/120a6e9f6c154a59ac6fa5901221936a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/120a6e9f6c154a59ac6fa5901221936a 2023-03-08 07:54:07,558 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/6409748420744c3d82a08bad2295f4ed to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/6409748420744c3d82a08bad2295f4ed 2023-03-08 07:54:07,560 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/64100875dd7f4679a8e23867c6bc07f4_SeqId_44_ 2023-03-08 07:54:07,583 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/.tmp/family1/65d9f4ff080e46f18839874a0136cd9d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/65d9f4ff080e46f18839874a0136cd9d 2023-03-08 07:54:07,590 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 of fbe4dd1fee45d9edc4c4fe869fc49ac1 into 65d9f4ff080e46f18839874a0136cd9d(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:54:07,590 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:07,591 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678262047492; duration=0sec 2023-03-08 07:54:07,591 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:07,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/ba7d06a745c04ce4a20c5a7d6ed533bf, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4583bc717488475a95b54dde0f2357f2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_] to archive 2023-03-08 07:54:07,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:07,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/ba7d06a745c04ce4a20c5a7d6ed533bf to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/ba7d06a745c04ce4a20c5a7d6ed533bf 2023-03-08 07:54:07,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4583bc717488475a95b54dde0f2357f2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4583bc717488475a95b54dde0f2357f2 2023-03-08 07:54:07,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/bce47919ac1c43f492e917599458126c_SeqId_47_ 2023-03-08 07:54:07,984 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130335cee59d7b147029a796fa4df1c073d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130335cee59d7b147029a796fa4df1c073d 2023-03-08 07:54:07,985 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:08,010 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0a472ccfaf0a4771a4e9163d9f088315 first=Optional[r00] last=Optional[r8] 2023-03-08 07:54:08,011 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0a472ccfaf0a4771a4e9163d9f088315}] 2023-03-08 07:54:08,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0a472ccfaf0a4771a4e9163d9f088315 for inclusion in fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:54:08,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 07:54:08,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:08,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:54:08,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0a472ccfaf0a4771a4e9163d9f088315 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7fhhfs62b4aenh6ssa5h4mtp6hn0th000rqpnj7q6cfqgqetpud0tqjscs2a8mj/family1/0a472ccfaf0a4771a4e9163d9f088315 2023-03-08 07:54:08,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7fhhfs62b4aenh6ssa5h4mtp6hn0th000rqpnj7q6cfqgqetpud0tqjscs2a8mj/family1/0a472ccfaf0a4771a4e9163d9f088315 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4d2dd27752ae4c128f8ba89554836fbd_SeqId_50_ 2023-03-08 07:54:08,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7fhhfs62b4aenh6ssa5h4mtp6hn0th000rqpnj7q6cfqgqetpud0tqjscs2a8mj/family1/0a472ccfaf0a4771a4e9163d9f088315 into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4d2dd27752ae4c128f8ba89554836fbd_SeqId_50_ - updating store file list. 2023-03-08 07:54:08,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4d2dd27752ae4c128f8ba89554836fbd_SeqId_50_ into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 2023-03-08 07:54:08,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7fhhfs62b4aenh6ssa5h4mtp6hn0th000rqpnj7q6cfqgqetpud0tqjscs2a8mj/family1/0a472ccfaf0a4771a4e9163d9f088315 into fbe4dd1fee45d9edc4c4fe869fc49ac1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/family1/4d2dd27752ae4c128f8ba89554836fbd_SeqId_50_) 2023-03-08 07:54:08,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7fhhfs62b4aenh6ssa5h4mtp6hn0th000rqpnj7q6cfqgqetpud0tqjscs2a8mj/family1/0a472ccfaf0a4771a4e9163d9f088315 2023-03-08 07:54:08,027 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:08,028 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:08,030 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112078ea3db9e4ed41bcb4c10abbc5bf870a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112078ea3db9e4ed41bcb4c10abbc5bf870a 2023-03-08 07:54:08,031 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128c949d79f83764e5190c7aa27c319d4de to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128c949d79f83764e5190c7aa27c319d4de 2023-03-08 07:54:08,032 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130bdfe4c4951154adc80b248ab840f37d3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130bdfe4c4951154adc80b248ab840f37d3 2023-03-08 07:54:08,032 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 07:54:08,032 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 07:54:08,032 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:08,032 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:08,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:54:08,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:54:08,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:08,034 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:08,035 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9490 sec 2023-03-08 07:54:08,086 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:54:08,086 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x165476fd to 127.0.0.1:52387 2023-03-08 07:54:08,086 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:54:08,087 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:54:08,087 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2205f02c to 127.0.0.1:52387 2023-03-08 07:54:08,087 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:54:08,088 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:54:08,088 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x59d158ad to 127.0.0.1:52387 2023-03-08 07:54:08,088 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:54:08,156 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249 (was 249), OpenFileDescriptor=600 (was 598) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=319 (was 351), ProcessCount=170 (was 170), AvailableMemoryMB=8066 (was 7462) - AvailableMemoryMB LEAK? - 2023-03-08 07:54:08,170 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=249, OpenFileDescriptor=600, MaxFileDescriptor=60000, SystemLoadAverage=319, ProcessCount=170, AvailableMemoryMB=8065 2023-03-08 07:54:08,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 07:54:08,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:08,176 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:54:08,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-08 07:54:08,177 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:54:08,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 07:54:08,179 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:08,180 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0 empty. 2023-03-08 07:54:08,180 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:08,180 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-08 07:54:08,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 07:54:08,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 07:54:08,591 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 07:54:08,592 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => a495beea80e2b5d5dc82eeff331974f0, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:08,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 07:54:08,998 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:08,998 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing a495beea80e2b5d5dc82eeff331974f0, disabling compactions & flushes 2023-03-08 07:54:08,999 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:08,999 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:08,999 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. after waiting 0 ms 2023-03-08 07:54:08,999 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:08,999 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:08,999 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:09,001 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:54:09,002 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678262049002"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262049002"}]},"ts":"1678262049002"} 2023-03-08 07:54:09,004 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:54:09,004 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:54:09,005 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262049005"}]},"ts":"1678262049005"} 2023-03-08 07:54:09,006 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 07:54:09,012 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a495beea80e2b5d5dc82eeff331974f0, ASSIGN}] 2023-03-08 07:54:09,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a495beea80e2b5d5dc82eeff331974f0, ASSIGN 2023-03-08 07:54:09,014 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a495beea80e2b5d5dc82eeff331974f0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:09,165 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=a495beea80e2b5d5dc82eeff331974f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:09,165 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678262049165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262049165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262049165"}]},"ts":"1678262049165"} 2023-03-08 07:54:09,167 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure a495beea80e2b5d5dc82eeff331974f0, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:09,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 07:54:09,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a495beea80e2b5d5dc82eeff331974f0, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,324 INFO [StoreOpener-a495beea80e2b5d5dc82eeff331974f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,326 DEBUG [StoreOpener-a495beea80e2b5d5dc82eeff331974f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1 2023-03-08 07:54:09,326 DEBUG [StoreOpener-a495beea80e2b5d5dc82eeff331974f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1 2023-03-08 07:54:09,326 INFO [StoreOpener-a495beea80e2b5d5dc82eeff331974f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a495beea80e2b5d5dc82eeff331974f0 columnFamilyName family1 2023-03-08 07:54:09,327 INFO [StoreOpener-a495beea80e2b5d5dc82eeff331974f0-1] regionserver.HStore(310): Store=a495beea80e2b5d5dc82eeff331974f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:09,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:54:09,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:09,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a495beea80e2b5d5dc82eeff331974f0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60210020, jitterRate=-0.10280078649520874}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:54:09,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:09,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0., pid=25, masterSystemTime=1678262049320 2023-03-08 07:54:09,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:09,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:09,337 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=a495beea80e2b5d5dc82eeff331974f0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:09,337 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678262049337"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262049337"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262049337"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262049337"}]},"ts":"1678262049337"} 2023-03-08 07:54:09,340 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-08 07:54:09,340 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure a495beea80e2b5d5dc82eeff331974f0, server=jenkins-hbase4.apache.org,33669,1678262016084 in 171 msec 2023-03-08 07:54:09,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-08 07:54:09,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a495beea80e2b5d5dc82eeff331974f0, ASSIGN in 328 msec 2023-03-08 07:54:09,343 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:54:09,343 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262049343"}]},"ts":"1678262049343"} 2023-03-08 07:54:09,344 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 07:54:09,346 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:54:09,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1740 sec 2023-03-08 07:54:10,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 07:54:10,284 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-08 07:54:10,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:10,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,311 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:10,311 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:10,311 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:10,311 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:10,312 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,312 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,313 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:10,313 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:10,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:10,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:10,313 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:10,313 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:10,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:10,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,314 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:10,315 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:10,315 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:10,316 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:54:10,739 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300feb1e29f10c49b783dcb07e021bbff6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511300feb1e29f10c49b783dcb07e021bbff6 2023-03-08 07:54:11,143 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/ba24020f973540b4a0b0a212532d75b2 2023-03-08 07:54:11,149 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/ba24020f973540b4a0b0a212532d75b2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ba24020f973540b4a0b0a212532d75b2 2023-03-08 07:54:11,154 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ba24020f973540b4a0b0a212532d75b2, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 07:54:11,155 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 838ms, sequenceid=5, compaction requested=false 2023-03-08 07:54:11,155 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:11,155 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:11,155 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:11,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:11,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:11,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:11,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:11,157 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,157 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:11,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:11,157 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,157 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:11,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:11,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:11,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:11,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:11,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f58caa7[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:11,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:11,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,160 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,160 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:11,160 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:11,160 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:11,160 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,160 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:11,162 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:11,162 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,162 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:11,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:11,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:11,162 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:11,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:11,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:11,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:11,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:11,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:11,164 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:11,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:11,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:11,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,164 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:11,165 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,165 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:11,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:11,165 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:11,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:11,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:11,166 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,167 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:11,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:11,169 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:11,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:11,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:11,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:11,170 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:11,170 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:11,270 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:11,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:11,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:11,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:11,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,277 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:11,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:11,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:11,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:11,278 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,278 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,279 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:11,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:11,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:11,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:11,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:11,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:11,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:11,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:11,281 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:11,281 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:11,281 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:11,281 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:11,281 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:11,281 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:11,281 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:11,282 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:54:11,695 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b66d74236b004d65a2f69efac1575510 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128b66d74236b004d65a2f69efac1575510 2023-03-08 07:54:12,101 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/2d460a13bcc04b9c968be779aa545d28 2023-03-08 07:54:12,107 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/2d460a13bcc04b9c968be779aa545d28 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d460a13bcc04b9c968be779aa545d28 2023-03-08 07:54:12,111 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d460a13bcc04b9c968be779aa545d28, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:54:12,112 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 830ms, sequenceid=9, compaction requested=false 2023-03-08 07:54:12,112 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:12,112 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:12,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:12,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:12,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:12,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:12,116 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,116 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:12,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:12,116 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,116 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:12,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:12,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:12,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:12,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:12,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14b9bcb5[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:12,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:12,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,118 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:12,118 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:12,118 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,118 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:12,118 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:12,120 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,120 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,120 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:12,120 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:12,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:12,122 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:12,122 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:12,122 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:12,122 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,122 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:12,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:12,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:12,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:12,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,124 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:12,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:12,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:12,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:12,125 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:12,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:12,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:12,128 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:12,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:12,128 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:12,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:12,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:12,228 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:12,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:12,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:12,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:12,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,235 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:12,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:12,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:12,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:12,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:12,237 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:12,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:12,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:12,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:12,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:12,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:12,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:12,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:12,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:12,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:12,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:12,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:12,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:12,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:12,239 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:12,651 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120125f80d9ea2194a4e9403d418ed990fc8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120125f80d9ea2194a4e9403d418ed990fc8 2023-03-08 07:54:13,057 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/e46cee4f0974489784c25bf48464d74b 2023-03-08 07:54:13,062 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/e46cee4f0974489784c25bf48464d74b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/e46cee4f0974489784c25bf48464d74b 2023-03-08 07:54:13,067 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/e46cee4f0974489784c25bf48464d74b, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 07:54:13,068 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 829ms, sequenceid=13, compaction requested=true 2023-03-08 07:54:13,068 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:13,068 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:13,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:13,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:13,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:13,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:13,070 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:13,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cf35553[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:13,071 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:13,072 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,073 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:13,073 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,073 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:13,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:13,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:13,075 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,075 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:13,075 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:13,075 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:13,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:13,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:13,077 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:13,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:13,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:13,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,077 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:13,082 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,082 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:13,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,082 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:13,082 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,083 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:13,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,085 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:13,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:13,085 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:13,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:13,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:13,086 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,086 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:13,086 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:13,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:13,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:13,186 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:13,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:13,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:13,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:13,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,198 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:13,198 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:13,198 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:13,198 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:13,198 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,198 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,201 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,201 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:13,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:13,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:13,201 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:13,201 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:13,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:13,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,202 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:13,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:13,202 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:13,202 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:13,202 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:13,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:13,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:13,203 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:13,203 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:13,203 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:13,621 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205517604c587dc41eaa54954d1856e6e1f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205517604c587dc41eaa54954d1856e6e1f 2023-03-08 07:54:14,027 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/f8923d562e3d481bb447eadfe572c3f8 2023-03-08 07:54:14,033 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/f8923d562e3d481bb447eadfe572c3f8 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f8923d562e3d481bb447eadfe572c3f8 2023-03-08 07:54:14,037 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f8923d562e3d481bb447eadfe572c3f8, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 07:54:14,038 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 835ms, sequenceid=17, compaction requested=true 2023-03-08 07:54:14,038 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:14,038 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:14,038 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:14,038 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:14,038 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,038 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:14,038 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:14,040 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,040 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:14,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:14,041 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,041 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:14,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:14,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:14,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:14,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:14,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56a99ffe[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:14,042 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:14,042 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,043 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,043 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:14,043 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:14,043 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:14,043 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,043 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:14,044 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,044 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:14,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:14,044 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:14,045 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:14,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:14,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:14,047 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:14,047 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:14,047 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:14,047 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,047 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:14,050 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,050 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:14,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:14,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,050 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:14,050 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:14,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:14,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:14,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:14,051 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:14,053 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:14,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:14,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:14,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:14,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:14,053 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:14,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:14,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,154 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:14,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:14,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:14,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:14,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,162 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:14,162 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:14,162 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:14,162 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:14,163 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,163 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,164 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:14,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:14,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:14,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,165 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:14,165 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:14,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:14,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:14,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:14,171 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:14,171 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:14,171 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:14,171 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:14,171 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:14,171 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:14,172 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:14,172 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:14,587 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512286d22e23650c24f7b8aefc04a726b35fb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512286d22e23650c24f7b8aefc04a726b35fb 2023-03-08 07:54:14,861 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:54:14,994 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/a42cd335b9c14d789bcc1a00f00f5b5e 2023-03-08 07:54:15,001 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/a42cd335b9c14d789bcc1a00f00f5b5e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a42cd335b9c14d789bcc1a00f00f5b5e 2023-03-08 07:54:15,008 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a42cd335b9c14d789bcc1a00f00f5b5e, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 07:54:15,009 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 837ms, sequenceid=21, compaction requested=true 2023-03-08 07:54:15,010 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:15,010 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:15,010 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:15,010 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:15,010 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,010 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:15,010 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:15,012 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,012 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:15,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:15,012 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,012 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:15,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:15,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:15,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:15,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:15,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f4d7a7e[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:15,014 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:15,014 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,015 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,015 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:15,015 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:15,015 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:15,015 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,015 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:15,017 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,017 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:15,017 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:15,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:15,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:15,017 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:15,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:15,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:15,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:15,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:15,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:15,019 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:15,019 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:15,019 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:15,020 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,020 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:15,022 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,022 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,022 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:15,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:15,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:15,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:15,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:15,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:15,023 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:15,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:15,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,027 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,027 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:15,027 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:15,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:15,027 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:15,027 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:15,028 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:15,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:15,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:15,128 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:15,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:15,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:15,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:15,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,140 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:15,140 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:15,140 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:15,140 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:15,141 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,141 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,142 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:15,142 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:15,143 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,143 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:15,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:15,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,144 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:15,144 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:15,144 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:15,145 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:15,145 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:15,145 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:15,145 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:15,145 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:15,564 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f3ecf442925e4d94958f774d98e33e97 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f3ecf442925e4d94958f774d98e33e97 2023-03-08 07:54:15,971 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/3eed8ae38bf5448e8c86eea278279398 2023-03-08 07:54:15,977 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/3eed8ae38bf5448e8c86eea278279398 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/3eed8ae38bf5448e8c86eea278279398 2023-03-08 07:54:15,983 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/3eed8ae38bf5448e8c86eea278279398, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 07:54:15,984 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 839ms, sequenceid=25, compaction requested=true 2023-03-08 07:54:15,985 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:15,985 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:15,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:15,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:15,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:15,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:15,986 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,987 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:15,987 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:15,987 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:15,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:15,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66c83536[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:15,989 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:15,989 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,991 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,991 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:15,991 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:15,991 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:15,992 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,992 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:15,996 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,996 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:15,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:15,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:15,996 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:15,996 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:15,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:15,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:15,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:15,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:15,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:15,999 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:15,999 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:15,999 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:15,999 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:15,999 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:16,001 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,001 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:16,001 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:16,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:16,002 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:16,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,003 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,006 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,006 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,006 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:16,007 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:16,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:16,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:16,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:16,007 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:16,008 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:16,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:16,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:16,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:16,108 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:16,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:16,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:16,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:16,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,117 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:16,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:16,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:16,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:16,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,119 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:16,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:16,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:16,119 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,119 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:16,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:16,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:16,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,121 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:16,121 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:16,121 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:16,122 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:16,122 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:16,122 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:16,122 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:16,122 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:16,148 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e51e9f87985c4a8ba5da89c0212471ed to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101e51e9f87985c4a8ba5da89c0212471ed 2023-03-08 07:54:16,158 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/5a6eb89a924340ffaf706c4cfa72d4de 2023-03-08 07:54:16,165 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/5a6eb89a924340ffaf706c4cfa72d4de as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/5a6eb89a924340ffaf706c4cfa72d4de 2023-03-08 07:54:16,171 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/5a6eb89a924340ffaf706c4cfa72d4de, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 07:54:16,172 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 50ms, sequenceid=29, compaction requested=true 2023-03-08 07:54:16,172 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:16,172 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:16,172 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:16,172 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:16,172 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,172 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:16,172 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:16,174 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,174 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:16,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:16,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:16,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:16,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:16,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77bd71[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:16,176 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:16,177 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,178 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,178 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:16,178 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:16,178 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:16,178 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,178 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:16,181 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,181 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:16,181 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:16,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,181 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:16,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:16,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:16,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:16,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:16,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:16,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:16,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:16,184 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:16,184 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:16,184 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:16,184 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,184 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:16,186 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,186 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:16,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,187 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:16,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:16,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:16,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:16,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:16,188 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:16,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,189 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:16,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,192 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:16,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:16,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:16,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:16,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:16,193 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:16,192 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,193 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:16,193 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,193 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,293 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:16,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:16,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:16,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:16,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,300 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:16,301 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:16,301 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:16,301 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:16,301 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,301 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,302 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,302 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:16,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:16,303 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:16,303 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:16,304 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:16,305 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:16,305 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:16,723 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035177bddfe19d4be9a3e9ff50e2a8fe63 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035177bddfe19d4be9a3e9ff50e2a8fe63 2023-03-08 07:54:17,130 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/2d792576befb4176b96b54e1396d98f9 2023-03-08 07:54:17,135 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/2d792576befb4176b96b54e1396d98f9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d792576befb4176b96b54e1396d98f9 2023-03-08 07:54:17,141 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d792576befb4176b96b54e1396d98f9, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 07:54:17,142 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 837ms, sequenceid=33, compaction requested=true 2023-03-08 07:54:17,142 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:17,142 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:17,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:17,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:17,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:17,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:17,145 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,145 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:17,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:17,145 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,145 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:17,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:17,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:17,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:17,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:17,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1abb84ee[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:17,147 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:17,147 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,148 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,148 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:17,148 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:17,148 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:17,148 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,148 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:17,149 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:17,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:17,152 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:17,152 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:17,152 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:17,152 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,152 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:17,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:17,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:17,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:17,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:17,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:17,154 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:17,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:17,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,155 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:17,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,159 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:17,159 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:17,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:17,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:17,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,160 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,160 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,160 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:17,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:17,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:17,160 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:17,260 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:17,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:17,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:17,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:17,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,269 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:17,269 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:17,269 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:17,269 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:17,270 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,270 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,271 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:17,271 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:17,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:17,271 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:17,271 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:17,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,272 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:17,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,273 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:17,273 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:17,273 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:17,273 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:17,273 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:17,273 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:17,687 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075b11b436ec2749cb8a49f10ddc53cb3d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509075b11b436ec2749cb8a49f10ddc53cb3d 2023-03-08 07:54:18,094 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca 2023-03-08 07:54:18,099 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca 2023-03-08 07:54:18,105 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 07:54:18,106 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 833ms, sequenceid=37, compaction requested=true 2023-03-08 07:54:18,106 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:18,106 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:18,106 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:18,106 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:18,106 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,106 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:18,106 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:18,108 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,108 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:18,108 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,108 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:18,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:18,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e71834d[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:18,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:18,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,115 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,116 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:18,116 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:18,116 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:18,116 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:18,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,118 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:18,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:18,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:18,118 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:18,118 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:18,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:18,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:18,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:18,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:18,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:18,121 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:18,121 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:18,121 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:18,121 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,121 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:18,122 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:18,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:18,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:18,123 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,124 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:18,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:18,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:18,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:18,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:18,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:18,127 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:18,127 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,128 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,128 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:18,128 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:18,228 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:18,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:18,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:18,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:18,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,235 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:18,235 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:18,235 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:18,235 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:18,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,236 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,238 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:18,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:18,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:18,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:18,239 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:18,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:18,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,240 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:18,240 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:18,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:18,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:18,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:18,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:18,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:18,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:18,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. started... 2023-03-08 07:54:18,240 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a495beea80e2b5d5dc82eeff331974f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:18,656 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120198dba09204346cc84c31a8f41405831 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120198dba09204346cc84c31a8f41405831 2023-03-08 07:54:19,064 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/2441aa8e78d04e1d8d2ac9357c28b368 2023-03-08 07:54:19,070 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/2441aa8e78d04e1d8d2ac9357c28b368 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2441aa8e78d04e1d8d2ac9357c28b368 2023-03-08 07:54:19,075 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2441aa8e78d04e1d8d2ac9357c28b368, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 07:54:19,076 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a495beea80e2b5d5dc82eeff331974f0 in 836ms, sequenceid=41, compaction requested=true 2023-03-08 07:54:19,077 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:19,077 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:19,077 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:19,077 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:19,077 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,077 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:19,077 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:19,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,079 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:19,079 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,079 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69b77e30[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:19,081 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:19,081 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,082 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:19,082 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,082 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:19,084 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,084 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:19,084 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:19,084 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:19,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:19,086 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:19,086 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:19,086 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:19,086 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,086 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:19,088 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,088 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:19,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,088 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:19,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:19,089 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:19,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,090 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:19,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:19,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 07:54:19,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:19,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:19,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:19,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:19,093 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:19,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:19,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,093 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,193 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:19,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:19,196 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-08 07:54:19,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:19,199 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:19,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:19,200 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262659200 2023-03-08 07:54:19,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 07:54:19,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 07:54:19,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:19,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x33de556b to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:19,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@647420be, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:19,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,670 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031751f4becae7469384612ebde54088a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601031751f4becae7469384612ebde54088a2 2023-03-08 07:54:19,671 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:19,684 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:54:19,686 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37930, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:54:19,692 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:54:19,694 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34764, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:54:19,708 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc first=Optional[r3] last=Optional[r6] 2023-03-08 07:54:19,709 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc}] 2023-03-08 07:54:19,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc for inclusion in a495beea80e2b5d5dc82eeff331974f0/family1 2023-03-08 07:54:19,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 07:54:19,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:19,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:19,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bjcdkeoa5ptc35mf4cbn4mmlnm94dfhqqqj58fr2t2bmiqjbiaaomjalfgne9dum/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc 2023-03-08 07:54:19,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bjcdkeoa5ptc35mf4cbn4mmlnm94dfhqqqj58fr2t2bmiqjbiaaomjalfgne9dum/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_ 2023-03-08 07:54:19,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bjcdkeoa5ptc35mf4cbn4mmlnm94dfhqqqj58fr2t2bmiqjbiaaomjalfgne9dum/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc into a495beea80e2b5d5dc82eeff331974f0/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_ - updating store file list. 2023-03-08 07:54:19,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_ into a495beea80e2b5d5dc82eeff331974f0/family1 2023-03-08 07:54:19,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bjcdkeoa5ptc35mf4cbn4mmlnm94dfhqqqj58fr2t2bmiqjbiaaomjalfgne9dum/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc into a495beea80e2b5d5dc82eeff331974f0/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_) 2023-03-08 07:54:19,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bjcdkeoa5ptc35mf4cbn4mmlnm94dfhqqqj58fr2t2bmiqjbiaaomjalfgne9dum/family1/6e07ad3a7d1b4327bfc11cbfe8cf22fc 2023-03-08 07:54:19,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:54:19,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:54:19,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): a495beea80e2b5d5dc82eeff331974f0/family1 is initiating minor compaction 2023-03-08 07:54:19,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6cf50a3f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:19,732 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a495beea80e2b5d5dc82eeff331974f0/family1 in testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:19,732 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d460a13bcc04b9c968be779aa545d28, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/e46cee4f0974489784c25bf48464d74b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f8923d562e3d481bb447eadfe572c3f8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a42cd335b9c14d789bcc1a00f00f5b5e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/3eed8ae38bf5448e8c86eea278279398, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/5a6eb89a924340ffaf706c4cfa72d4de, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d792576befb4176b96b54e1396d98f9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2441aa8e78d04e1d8d2ac9357c28b368, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp, totalSize=50.0 K 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 2d460a13bcc04b9c968be779aa545d28, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting e46cee4f0974489784c25bf48464d74b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting f8923d562e3d481bb447eadfe572c3f8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting a42cd335b9c14d789bcc1a00f00f5b5e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 3eed8ae38bf5448e8c86eea278279398, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 5a6eb89a924340ffaf706c4cfa72d4de, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 2d792576befb4176b96b54e1396d98f9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting b7cdfc87e25c4b3fb9bd4bfa17a1f7ca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 2441aa8e78d04e1d8d2ac9357c28b368, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:54:19,732 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:54:19,734 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:19,735 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:19,738 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512286d22e23650c24f7b8aefc04a726b35fb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512286d22e23650c24f7b8aefc04a726b35fb 2023-03-08 07:54:19,739 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f3ecf442925e4d94958f774d98e33e97 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f3ecf442925e4d94958f774d98e33e97 2023-03-08 07:54:19,740 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101e51e9f87985c4a8ba5da89c0212471ed to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101e51e9f87985c4a8ba5da89c0212471ed 2023-03-08 07:54:19,742 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035177bddfe19d4be9a3e9ff50e2a8fe63 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035177bddfe19d4be9a3e9ff50e2a8fe63 2023-03-08 07:54:19,742 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 07:54:19,742 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:19,795 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): a495beea80e2b5d5dc82eeff331974f0#family1#compaction#34 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:54:20,605 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054c755bb1d8154327bf2582aa7f010d5f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054c755bb1d8154327bf2582aa7f010d5f 2023-03-08 07:54:20,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:20,617 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/c2bad527207846cdb3466a6a91f08b5c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/c2bad527207846cdb3466a6a91f08b5c 2023-03-08 07:54:20,624 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a495beea80e2b5d5dc82eeff331974f0/family1 of a495beea80e2b5d5dc82eeff331974f0 into c2bad527207846cdb3466a6a91f08b5c(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:54:20,624 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:20,624 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678262059731; duration=0sec 2023-03-08 07:54:20,624 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:20,633 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c5ea5edf1d9f42b6a4e268a368eab29d first=Optional[r1] last=Optional[r2] 2023-03-08 07:54:20,633 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c5ea5edf1d9f42b6a4e268a368eab29d}] 2023-03-08 07:54:20,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c5ea5edf1d9f42b6a4e268a368eab29d for inclusion in a495beea80e2b5d5dc82eeff331974f0/family1 2023-03-08 07:54:20,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 07:54:20,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:20,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:20,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c5ea5edf1d9f42b6a4e268a368eab29d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pn9f5lv2av1r3366ogd2a5gjehkj70s73hc911k74cc7rdsjdkb6fe5nvm5ef6ng/family1/c5ea5edf1d9f42b6a4e268a368eab29d 2023-03-08 07:54:20,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pn9f5lv2av1r3366ogd2a5gjehkj70s73hc911k74cc7rdsjdkb6fe5nvm5ef6ng/family1/c5ea5edf1d9f42b6a4e268a368eab29d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_ 2023-03-08 07:54:20,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pn9f5lv2av1r3366ogd2a5gjehkj70s73hc911k74cc7rdsjdkb6fe5nvm5ef6ng/family1/c5ea5edf1d9f42b6a4e268a368eab29d into a495beea80e2b5d5dc82eeff331974f0/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_ - updating store file list. 2023-03-08 07:54:20,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_ into a495beea80e2b5d5dc82eeff331974f0/family1 2023-03-08 07:54:20,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pn9f5lv2av1r3366ogd2a5gjehkj70s73hc911k74cc7rdsjdkb6fe5nvm5ef6ng/family1/c5ea5edf1d9f42b6a4e268a368eab29d into a495beea80e2b5d5dc82eeff331974f0/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_) 2023-03-08 07:54:20,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pn9f5lv2av1r3366ogd2a5gjehkj70s73hc911k74cc7rdsjdkb6fe5nvm5ef6ng/family1/c5ea5edf1d9f42b6a4e268a368eab29d 2023-03-08 07:54:20,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 07:54:20,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 07:54:20,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): a495beea80e2b5d5dc82eeff331974f0/family1 is initiating minor compaction (all files) 2023-03-08 07:54:20,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@657be90f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:20,651 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a495beea80e2b5d5dc82eeff331974f0/family1 in testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:54:20,652 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ba24020f973540b4a0b0a212532d75b2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/c2bad527207846cdb3466a6a91f08b5c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp, totalSize=176.8 K 2023-03-08 07:54:20,652 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ba24020f973540b4a0b0a212532d75b2, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839991 2023-03-08 07:54:20,653 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting c2bad527207846cdb3466a6a91f08b5c, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239991 2023-03-08 07:54:20,653 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ce2ce52e0a444430abb8211727b312ac_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239991 2023-03-08 07:54:20,654 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:20,654 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:20,656 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120125f80d9ea2194a4e9403d418ed990fc8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120125f80d9ea2194a4e9403d418ed990fc8 2023-03-08 07:54:20,658 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205517604c587dc41eaa54954d1856e6e1f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205517604c587dc41eaa54954d1856e6e1f 2023-03-08 07:54:20,658 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-08 07:54:20,658 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 07:54:20,658 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:20,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:20,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:20,660 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:20,661 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4630 sec 2023-03-08 07:54:20,672 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): a495beea80e2b5d5dc82eeff331974f0#family1#compaction#35 average throughput is 19.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:54:20,728 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d460a13bcc04b9c968be779aa545d28, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/e46cee4f0974489784c25bf48464d74b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f8923d562e3d481bb447eadfe572c3f8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a42cd335b9c14d789bcc1a00f00f5b5e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/3eed8ae38bf5448e8c86eea278279398, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/5a6eb89a924340ffaf706c4cfa72d4de, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d792576befb4176b96b54e1396d98f9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2441aa8e78d04e1d8d2ac9357c28b368, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_] to archive 2023-03-08 07:54:20,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:20,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d460a13bcc04b9c968be779aa545d28 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d460a13bcc04b9c968be779aa545d28 2023-03-08 07:54:20,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/e46cee4f0974489784c25bf48464d74b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/e46cee4f0974489784c25bf48464d74b 2023-03-08 07:54:20,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f8923d562e3d481bb447eadfe572c3f8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f8923d562e3d481bb447eadfe572c3f8 2023-03-08 07:54:20,736 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a42cd335b9c14d789bcc1a00f00f5b5e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a42cd335b9c14d789bcc1a00f00f5b5e 2023-03-08 07:54:20,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/3eed8ae38bf5448e8c86eea278279398 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/3eed8ae38bf5448e8c86eea278279398 2023-03-08 07:54:20,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/5a6eb89a924340ffaf706c4cfa72d4de to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/5a6eb89a924340ffaf706c4cfa72d4de 2023-03-08 07:54:20,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d792576befb4176b96b54e1396d98f9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2d792576befb4176b96b54e1396d98f9 2023-03-08 07:54:20,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/b7cdfc87e25c4b3fb9bd4bfa17a1f7ca 2023-03-08 07:54:20,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2441aa8e78d04e1d8d2ac9357c28b368 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/2441aa8e78d04e1d8d2ac9357c28b368 2023-03-08 07:54:20,747 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/f3a62d85d6dc4787b52d338a0e5de880_SeqId_44_ 2023-03-08 07:54:20,762 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=255 (was 249) - Thread LEAK? -, OpenFileDescriptor=601 (was 600) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=293 (was 319), ProcessCount=170 (was 170), AvailableMemoryMB=7529 (was 8065) 2023-03-08 07:54:20,773 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255, OpenFileDescriptor=601, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=170, AvailableMemoryMB=7529 2023-03-08 07:54:20,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 07:54:20,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:20,779 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:54:20,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-08 07:54:20,779 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:54:20,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 07:54:20,782 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:20,782 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:20,782 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:20,782 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944 empty. 2023-03-08 07:54:20,782 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903 empty. 2023-03-08 07:54:20,782 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c empty. 2023-03-08 07:54:20,783 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:20,783 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:20,783 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:20,783 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-08 07:54:20,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 07:54:21,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 07:54:21,192 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-08 07:54:21,194 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 4a2a1f49d57eee4dc8b34f34c4c02a5c, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:21,194 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 806d93567d24c9f9a1e9b8e642f88903, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:21,194 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3cf54bc577fa89b1f72695aed8375944, NAME => 'testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:21,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 07:54:21,609 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:21,609 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:21,609 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 806d93567d24c9f9a1e9b8e642f88903, disabling compactions & flushes 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 4a2a1f49d57eee4dc8b34f34c4c02a5c, disabling compactions & flushes 2023-03-08 07:54:21,610 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,610 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. after waiting 0 ms 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. after waiting 0 ms 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 3cf54bc577fa89b1f72695aed8375944, disabling compactions & flushes 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,610 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,610 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:54:21,610 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,610 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. after waiting 0 ms 2023-03-08 07:54:21,611 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,611 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 806d93567d24c9f9a1e9b8e642f88903: 2023-03-08 07:54:21,611 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,611 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 3cf54bc577fa89b1f72695aed8375944: 2023-03-08 07:54:21,613 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:54:21,614 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678262061614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262061614"}]},"ts":"1678262061614"} 2023-03-08 07:54:21,614 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678262061614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262061614"}]},"ts":"1678262061614"} 2023-03-08 07:54:21,614 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678262061614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262061614"}]},"ts":"1678262061614"} 2023-03-08 07:54:21,621 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 07:54:21,622 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:54:21,622 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262061622"}]},"ts":"1678262061622"} 2023-03-08 07:54:21,623 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-08 07:54:21,627 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3cf54bc577fa89b1f72695aed8375944, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=806d93567d24c9f9a1e9b8e642f88903, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a2a1f49d57eee4dc8b34f34c4c02a5c, ASSIGN}] 2023-03-08 07:54:21,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=806d93567d24c9f9a1e9b8e642f88903, ASSIGN 2023-03-08 07:54:21,629 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a2a1f49d57eee4dc8b34f34c4c02a5c, ASSIGN 2023-03-08 07:54:21,629 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3cf54bc577fa89b1f72695aed8375944, ASSIGN 2023-03-08 07:54:21,630 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3cf54bc577fa89b1f72695aed8375944, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:21,630 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=806d93567d24c9f9a1e9b8e642f88903, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:21,630 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a2a1f49d57eee4dc8b34f34c4c02a5c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:21,781 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=3cf54bc577fa89b1f72695aed8375944, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:21,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=806d93567d24c9f9a1e9b8e642f88903, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:21,781 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=4a2a1f49d57eee4dc8b34f34c4c02a5c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:21,782 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678262061781"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262061781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262061781"}]},"ts":"1678262061781"} 2023-03-08 07:54:21,782 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678262061781"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262061781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262061781"}]},"ts":"1678262061781"} 2023-03-08 07:54:21,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678262061781"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262061781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262061781"}]},"ts":"1678262061781"} 2023-03-08 07:54:21,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 4a2a1f49d57eee4dc8b34f34c4c02a5c, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:21,785 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 3cf54bc577fa89b1f72695aed8375944, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:21,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 806d93567d24c9f9a1e9b8e642f88903, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:21,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 07:54:21,900 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/.tmp/family1/a45c4e13516a413c974a4a462ef7daac as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/a45c4e13516a413c974a4a462ef7daac 2023-03-08 07:54:21,906 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a495beea80e2b5d5dc82eeff331974f0/family1 of a495beea80e2b5d5dc82eeff331974f0 into a45c4e13516a413c974a4a462ef7daac(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:54:21,906 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:54:21,907 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1678262060651; duration=1sec 2023-03-08 07:54:21,907 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:21,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ba24020f973540b4a0b0a212532d75b2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/c2bad527207846cdb3466a6a91f08b5c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_] to archive 2023-03-08 07:54:21,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:21,924 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ba24020f973540b4a0b0a212532d75b2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ba24020f973540b4a0b0a212532d75b2 2023-03-08 07:54:21,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/c2bad527207846cdb3466a6a91f08b5c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/c2bad527207846cdb3466a6a91f08b5c 2023-03-08 07:54:21,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/family1/ce2ce52e0a444430abb8211727b312ac_SeqId_47_ 2023-03-08 07:54:21,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 806d93567d24c9f9a1e9b8e642f88903, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 07:54:21,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:21,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,941 INFO [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,942 DEBUG [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903/family1 2023-03-08 07:54:21,942 DEBUG [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903/family1 2023-03-08 07:54:21,942 INFO [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 806d93567d24c9f9a1e9b8e642f88903 columnFamilyName family1 2023-03-08 07:54:21,944 INFO [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] regionserver.HStore(310): Store=806d93567d24c9f9a1e9b8e642f88903/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:21,944 INFO [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,946 DEBUG [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903/family2 2023-03-08 07:54:21,946 DEBUG [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903/family2 2023-03-08 07:54:21,946 INFO [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 806d93567d24c9f9a1e9b8e642f88903 columnFamilyName family2 2023-03-08 07:54:21,948 INFO [StoreOpener-806d93567d24c9f9a1e9b8e642f88903-1] regionserver.HStore(310): Store=806d93567d24c9f9a1e9b8e642f88903/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:21,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:54:21,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:54:21,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:21,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 806d93567d24c9f9a1e9b8e642f88903; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68159421, jitterRate=0.015654519200325012}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:54:21,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 806d93567d24c9f9a1e9b8e642f88903: 2023-03-08 07:54:21,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903., pid=33, masterSystemTime=1678262061936 2023-03-08 07:54:21,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:21,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3cf54bc577fa89b1f72695aed8375944, NAME => 'testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 07:54:21,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:21,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,958 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=806d93567d24c9f9a1e9b8e642f88903, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:21,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,958 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678262061957"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262061957"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262061957"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262061957"}]},"ts":"1678262061957"} 2023-03-08 07:54:21,959 INFO [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,960 DEBUG [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944/family1 2023-03-08 07:54:21,960 DEBUG [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944/family1 2023-03-08 07:54:21,961 INFO [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3cf54bc577fa89b1f72695aed8375944 columnFamilyName family1 2023-03-08 07:54:21,961 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-08 07:54:21,961 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 806d93567d24c9f9a1e9b8e642f88903, server=jenkins-hbase4.apache.org,33669,1678262016084 in 173 msec 2023-03-08 07:54:21,962 INFO [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] regionserver.HStore(310): Store=3cf54bc577fa89b1f72695aed8375944/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:21,962 INFO [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=806d93567d24c9f9a1e9b8e642f88903, ASSIGN in 334 msec 2023-03-08 07:54:21,963 DEBUG [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944/family2 2023-03-08 07:54:21,964 DEBUG [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944/family2 2023-03-08 07:54:21,964 INFO [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3cf54bc577fa89b1f72695aed8375944 columnFamilyName family2 2023-03-08 07:54:21,965 INFO [StoreOpener-3cf54bc577fa89b1f72695aed8375944-1] regionserver.HStore(310): Store=3cf54bc577fa89b1f72695aed8375944/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:21,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:54:21,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:54:21,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:21,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3cf54bc577fa89b1f72695aed8375944; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60850876, jitterRate=-0.0932512879371643}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:54:21,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3cf54bc577fa89b1f72695aed8375944: 2023-03-08 07:54:21,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944., pid=32, masterSystemTime=1678262061936 2023-03-08 07:54:21,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4a2a1f49d57eee4dc8b34f34c4c02a5c, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 07:54:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,974 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=3cf54bc577fa89b1f72695aed8375944, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,974 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678262061974"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262061974"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262061974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262061974"}]},"ts":"1678262061974"} 2023-03-08 07:54:21,975 INFO [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,978 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-03-08 07:54:21,978 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 3cf54bc577fa89b1f72695aed8375944, server=jenkins-hbase4.apache.org,33669,1678262016084 in 190 msec 2023-03-08 07:54:21,979 DEBUG [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 2023-03-08 07:54:21,979 DEBUG [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 2023-03-08 07:54:21,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3cf54bc577fa89b1f72695aed8375944, ASSIGN in 351 msec 2023-03-08 07:54:21,979 INFO [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a2a1f49d57eee4dc8b34f34c4c02a5c columnFamilyName family1 2023-03-08 07:54:21,980 INFO [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] regionserver.HStore(310): Store=4a2a1f49d57eee4dc8b34f34c4c02a5c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:21,980 INFO [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,982 DEBUG [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family2 2023-03-08 07:54:21,982 DEBUG [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family2 2023-03-08 07:54:21,982 INFO [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a2a1f49d57eee4dc8b34f34c4c02a5c columnFamilyName family2 2023-03-08 07:54:21,983 INFO [StoreOpener-4a2a1f49d57eee4dc8b34f34c4c02a5c-1] regionserver.HStore(310): Store=4a2a1f49d57eee4dc8b34f34c4c02a5c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:21,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:54:21,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:54:21,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:21,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4a2a1f49d57eee4dc8b34f34c4c02a5c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61675552, jitterRate=-0.0809626579284668}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:54:21,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:54:21,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c., pid=31, masterSystemTime=1678262061936 2023-03-08 07:54:21,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:21,994 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=4a2a1f49d57eee4dc8b34f34c4c02a5c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:21,994 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678262061994"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262061994"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262061994"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262061994"}]},"ts":"1678262061994"} 2023-03-08 07:54:21,998 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-03-08 07:54:21,998 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 4a2a1f49d57eee4dc8b34f34c4c02a5c, server=jenkins-hbase4.apache.org,33669,1678262016084 in 213 msec 2023-03-08 07:54:22,000 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-08 07:54:22,000 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a2a1f49d57eee4dc8b34f34c4c02a5c, ASSIGN in 371 msec 2023-03-08 07:54:22,001 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:54:22,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262062001"}]},"ts":"1678262062001"} 2023-03-08 07:54:22,003 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-08 07:54:22,005 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:54:22,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2290 sec 2023-03-08 07:54:22,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 07:54:22,887 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-08 07:54:22,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:22,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,896 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 07:54:22,896 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:22,897 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 07:54:22,897 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 07:54:22,897 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,897 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,898 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:22,898 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:22,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:22,898 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:22,899 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,899 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:22,900 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 07:54:22,900 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:22,900 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 07:54:22,900 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:22,900 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. started... 2023-03-08 07:54:22,900 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:22,901 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:54:22,901 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:22,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:22,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3cf54bc577fa89b1f72695aed8375944: 2023-03-08 07:54:22,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. started... 2023-03-08 07:54:22,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:22,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. started... 2023-03-08 07:54:22,902 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:54:22,902 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 4a2a1f49d57eee4dc8b34f34c4c02a5c 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-08 07:54:22,902 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806d93567d24c9f9a1e9b8e642f88903: 2023-03-08 07:54:22,902 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:22,902 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:54:23,318 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303082130bb08040c48209ef5d647d895c907 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303082130bb08040c48209ef5d647d895c907 2023-03-08 07:54:23,723 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/.tmp/family1/88db2c93592b4be3b7bab3fd9a12a122 2023-03-08 07:54:23,728 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/.tmp/family1/88db2c93592b4be3b7bab3fd9a12a122 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/88db2c93592b4be3b7bab3fd9a12a122 2023-03-08 07:54:23,733 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/88db2c93592b4be3b7bab3fd9a12a122, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 07:54:23,734 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 4a2a1f49d57eee4dc8b34f34c4c02a5c in 832ms, sequenceid=5, compaction requested=false 2023-03-08 07:54:23,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-08 07:54:23,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:54:23,735 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:23,735 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:54:23,735 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:54:23,735 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,735 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 07:54:23,735 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 07:54:23,737 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,737 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:23,738 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,738 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:23,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 07:54:23,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5496da42[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:23,739 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 07:54:23,739 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,741 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,741 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 07:54:23,741 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,741 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:23,741 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 07:54:23,741 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:23,743 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 07:54:23,743 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,743 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:23,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:23,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:23,743 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 07:54:23,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:23,745 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 07:54:23,745 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 07:54:23,745 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:23,745 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:23,745 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,745 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:23,747 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,747 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,747 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:23,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:23,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:23,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,747 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:23,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:23,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:23,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:23,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,749 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:23,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:23,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 07:54:23,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:23,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:23,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:23,757 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:23,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:23,757 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:23,758 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,758 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,758 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,858 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:23,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:23,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:23,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:23,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,865 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 07:54:23,865 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:23,865 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 07:54:23,865 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 07:54:23,865 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,866 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,867 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:23,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:23,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:23,867 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:23,867 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:23,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:23,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-08 07:54:23,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:23,868 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 07:54:23,868 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:23,868 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 07:54:23,868 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. started... 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3cf54bc577fa89b1f72695aed8375944: 2023-03-08 07:54:23,869 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:54:23,869 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:23,869 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. started... 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. started... 2023-03-08 07:54:23,869 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 4a2a1f49d57eee4dc8b34f34c4c02a5c 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806d93567d24c9f9a1e9b8e642f88903: 2023-03-08 07:54:23,869 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:54:23,869 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:54:24,285 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303084cce58aec25644e49acb8be6a45bbcb6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303084cce58aec25644e49acb8be6a45bbcb6 2023-03-08 07:54:24,690 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/.tmp/family1/8b28a00158b24ea39fcfb362eb9d55cd 2023-03-08 07:54:24,696 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/.tmp/family1/8b28a00158b24ea39fcfb362eb9d55cd as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b28a00158b24ea39fcfb362eb9d55cd 2023-03-08 07:54:24,701 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b28a00158b24ea39fcfb362eb9d55cd, entries=2, sequenceid=9, filesize=5.0 K 2023-03-08 07:54:24,702 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 4a2a1f49d57eee4dc8b34f34c4c02a5c in 833ms, sequenceid=9, compaction requested=false 2023-03-08 07:54:24,703 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:54:24,703 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:24,703 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:54:24,703 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:54:24,703 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,703 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 07:54:24,704 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 07:54:24,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:24,706 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:24,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:24,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:24,706 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,706 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:24,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 07:54:24,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@124ca8ca[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:24,708 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 07:54:24,708 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,709 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,709 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 07:54:24,709 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,709 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:24,709 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 07:54:24,709 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:24,711 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 07:54:24,711 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,711 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:24,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:24,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:24,711 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 07:54:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:24,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:24,713 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 07:54:24,713 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 07:54:24,713 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:24,713 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:24,713 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,713 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:24,715 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,715 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:24,715 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,715 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:24,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:24,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,717 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:24,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,719 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:24,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-08 07:54:24,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:24,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:24,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:24,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 07:54:24,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:24,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:24,720 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,720 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,720 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,720 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:24,720 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:24,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:24,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:24,820 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:24,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:24,828 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-08 07:54:24,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 07:54:24,830 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 07:54:24,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:24,831 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262664831 2023-03-08 07:54:24,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:54:24,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-08 07:54:24,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 07:54:24,833 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x60424cf6 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:24,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@25f6f190, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:24,837 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 07:54:25,665 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308a9b1d37ac3084abc8d4bf6a5895b5140 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308a9b1d37ac3084abc8d4bf6a5895b5140 2023-03-08 07:54:25,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 07:54:25,678 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:54:25,680 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54830, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:54:25,688 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:54:25,691 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57244, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:54:25,701 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/835bfcbf268b45d9bc91efb045e3ceac first=Optional[k0] last=Optional[k1] 2023-03-08 07:54:25,701 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/835bfcbf268b45d9bc91efb045e3ceac}] 2023-03-08 07:54:25,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/835bfcbf268b45d9bc91efb045e3ceac for inclusion in 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 2023-03-08 07:54:25,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-08 07:54:25,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 07:54:25,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:54:25,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/835bfcbf268b45d9bc91efb045e3ceac to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testScannerOnBulkLoadRefHFiles__2uel94d64qrli0oufnfhlj3k22dfdfpn84e33isgvpblg4je0h0kplmv90mneum1/family1/835bfcbf268b45d9bc91efb045e3ceac 2023-03-08 07:54:25,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testScannerOnBulkLoadRefHFiles__2uel94d64qrli0oufnfhlj3k22dfdfpn84e33isgvpblg4je0h0kplmv90mneum1/family1/835bfcbf268b45d9bc91efb045e3ceac as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_ 2023-03-08 07:54:25,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testScannerOnBulkLoadRefHFiles__2uel94d64qrli0oufnfhlj3k22dfdfpn84e33isgvpblg4je0h0kplmv90mneum1/family1/835bfcbf268b45d9bc91efb045e3ceac into 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_ - updating store file list. 2023-03-08 07:54:25,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_ into 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 2023-03-08 07:54:25,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testScannerOnBulkLoadRefHFiles__2uel94d64qrli0oufnfhlj3k22dfdfpn84e33isgvpblg4je0h0kplmv90mneum1/family1/835bfcbf268b45d9bc91efb045e3ceac into 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_) 2023-03-08 07:54:25,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testScannerOnBulkLoadRefHFiles__2uel94d64qrli0oufnfhlj3k22dfdfpn84e33isgvpblg4je0h0kplmv90mneum1/family1/835bfcbf268b45d9bc91efb045e3ceac 2023-03-08 07:54:25,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 07:54:25,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 07:54:25,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 is initiating minor compaction (all files) 2023-03-08 07:54:25,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6df3ceb7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:25,719 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 in testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:54:25,719 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/88db2c93592b4be3b7bab3fd9a12a122, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b28a00158b24ea39fcfb362eb9d55cd, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/.tmp, totalSize=15.1 K 2023-03-08 07:54:25,719 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 88db2c93592b4be3b7bab3fd9a12a122, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262062888 2023-03-08 07:54:25,720 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 8b28a00158b24ea39fcfb362eb9d55cd, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262062888 2023-03-08 07:54:25,720 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 95af27c7992f446ba0c051d7e06fc796_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262062888 2023-03-08 07:54:25,721 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 07:54:25,721 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:25,723 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303082130bb08040c48209ef5d647d895c907 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303082130bb08040c48209ef5d647d895c907 2023-03-08 07:54:25,725 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303084cce58aec25644e49acb8be6a45bbcb6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303084cce58aec25644e49acb8be6a45bbcb6 2023-03-08 07:54:25,725 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-08 07:54:25,725 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 07:54:25,725 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 07:54:25,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 07:54:25,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:25,727 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 07:54:25,728 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 898 msec 2023-03-08 07:54:25,731 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4a2a1f49d57eee4dc8b34f34c4c02a5c#family1#compaction#38 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:54:25,768 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=252 (was 255), OpenFileDescriptor=606 (was 601) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=278 (was 293), ProcessCount=170 (was 170), AvailableMemoryMB=7602 (was 7529) - AvailableMemoryMB LEAK? - 2023-03-08 07:54:25,779 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=252, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=278, ProcessCount=170, AvailableMemoryMB=7602 2023-03-08 07:54:25,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 07:54:25,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:25,783 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:54:25,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-08 07:54:25,784 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:54:25,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 07:54:25,786 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:25,786 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3 empty. 2023-03-08 07:54:25,787 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:25,787 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-08 07:54:25,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 07:54:26,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 07:54:26,198 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 07:54:26,199 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => e906f28e8a2d2e81f61b506d78f3c6d3, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:26,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 07:54:26,605 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:26,605 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing e906f28e8a2d2e81f61b506d78f3c6d3, disabling compactions & flushes 2023-03-08 07:54:26,605 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,605 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,605 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. after waiting 0 ms 2023-03-08 07:54:26,605 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,605 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,605 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:26,608 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:54:26,609 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678262066609"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262066609"}]},"ts":"1678262066609"} 2023-03-08 07:54:26,610 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:54:26,611 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:54:26,611 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262066611"}]},"ts":"1678262066611"} 2023-03-08 07:54:26,612 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 07:54:26,616 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e906f28e8a2d2e81f61b506d78f3c6d3, ASSIGN}] 2023-03-08 07:54:26,617 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e906f28e8a2d2e81f61b506d78f3c6d3, ASSIGN 2023-03-08 07:54:26,618 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e906f28e8a2d2e81f61b506d78f3c6d3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:26,769 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=e906f28e8a2d2e81f61b506d78f3c6d3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:26,769 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678262066769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262066769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262066769"}]},"ts":"1678262066769"} 2023-03-08 07:54:26,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure e906f28e8a2d2e81f61b506d78f3c6d3, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:26,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 07:54:26,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e906f28e8a2d2e81f61b506d78f3c6d3, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:54:26,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:26,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,927 INFO [StoreOpener-e906f28e8a2d2e81f61b506d78f3c6d3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,929 DEBUG [StoreOpener-e906f28e8a2d2e81f61b506d78f3c6d3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1 2023-03-08 07:54:26,929 DEBUG [StoreOpener-e906f28e8a2d2e81f61b506d78f3c6d3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1 2023-03-08 07:54:26,929 INFO [StoreOpener-e906f28e8a2d2e81f61b506d78f3c6d3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e906f28e8a2d2e81f61b506d78f3c6d3 columnFamilyName family1 2023-03-08 07:54:26,931 INFO [StoreOpener-e906f28e8a2d2e81f61b506d78f3c6d3-1] regionserver.HStore(310): Store=e906f28e8a2d2e81f61b506d78f3c6d3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:26,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:54:26,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:26,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e906f28e8a2d2e81f61b506d78f3c6d3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70335468, jitterRate=0.04808014631271362}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:54:26,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:26,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3., pid=37, masterSystemTime=1678262066922 2023-03-08 07:54:26,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:26,941 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=e906f28e8a2d2e81f61b506d78f3c6d3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:26,941 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678262066941"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262066941"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262066941"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262066941"}]},"ts":"1678262066941"} 2023-03-08 07:54:26,944 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-08 07:54:26,944 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure e906f28e8a2d2e81f61b506d78f3c6d3, server=jenkins-hbase4.apache.org,33669,1678262016084 in 171 msec 2023-03-08 07:54:26,946 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-08 07:54:26,946 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e906f28e8a2d2e81f61b506d78f3c6d3, ASSIGN in 328 msec 2023-03-08 07:54:26,947 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:54:26,947 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262066947"}]},"ts":"1678262066947"} 2023-03-08 07:54:26,949 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 07:54:26,952 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:54:26,954 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1710 sec 2023-03-08 07:54:26,955 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/.tmp/family1/8b2888c99f4b4929be6970177f61013a as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b2888c99f4b4929be6970177f61013a 2023-03-08 07:54:26,961 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 4a2a1f49d57eee4dc8b34f34c4c02a5c/family1 of 4a2a1f49d57eee4dc8b34f34c4c02a5c into 8b2888c99f4b4929be6970177f61013a(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:54:26,961 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:54:26,961 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678262065718; duration=1sec 2023-03-08 07:54:26,961 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:27,022 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/88db2c93592b4be3b7bab3fd9a12a122, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b28a00158b24ea39fcfb362eb9d55cd, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_] to archive 2023-03-08 07:54:27,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:27,024 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/88db2c93592b4be3b7bab3fd9a12a122 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/88db2c93592b4be3b7bab3fd9a12a122 2023-03-08 07:54:27,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b28a00158b24ea39fcfb362eb9d55cd to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/8b28a00158b24ea39fcfb362eb9d55cd 2023-03-08 07:54:27,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/family1/95af27c7992f446ba0c051d7e06fc796_SeqId_12_ 2023-03-08 07:54:27,472 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:54:27,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 07:54:27,890 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-08 07:54:27,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:27,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,917 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:27,917 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:27,918 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:27,918 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:27,918 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,918 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,919 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:27,919 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:27,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:27,920 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:27,920 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:27,921 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:27,921 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:27,922 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:54:28,340 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511308859e851d4e94229a2d6d651b230b323 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511308859e851d4e94229a2d6d651b230b323 2023-03-08 07:54:28,744 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/ae2f1aa60bab489abb15ce8ac56b99ad 2023-03-08 07:54:28,749 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/ae2f1aa60bab489abb15ce8ac56b99ad as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/ae2f1aa60bab489abb15ce8ac56b99ad 2023-03-08 07:54:28,753 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/ae2f1aa60bab489abb15ce8ac56b99ad, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 07:54:28,754 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 833ms, sequenceid=5, compaction requested=false 2023-03-08 07:54:28,754 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:28,755 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:28,755 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:28,758 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,758 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:28,758 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,758 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:28,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:28,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:28,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50653a25[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:28,759 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:28,759 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,761 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,761 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:28,761 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:28,761 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:28,761 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,761 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:28,762 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:28,762 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,762 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:28,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:28,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:28,762 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:28,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:28,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:28,765 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:28,765 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:28,765 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:28,765 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,765 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:28,766 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,766 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:28,766 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:28,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,767 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:28,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:28,768 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:28,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:28,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:28,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:28,771 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:28,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:28,771 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,772 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:28,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:28,772 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,772 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:28,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:28,872 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:28,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:28,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:28,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:28,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,879 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:28,879 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:28,879 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:28,879 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:28,880 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,880 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,881 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,881 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:28,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:28,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:28,881 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:28,881 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:28,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:28,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,883 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:28,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:28,883 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:28,883 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:28,883 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:28,883 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:28,883 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:28,884 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:28,884 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:28,884 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:54:29,297 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282540c6df581546e8ac63c64a7e5c81a3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511282540c6df581546e8ac63c64a7e5c81a3 2023-03-08 07:54:29,704 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/84a8a86772a44056974bf4744a3cf557 2023-03-08 07:54:29,708 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/84a8a86772a44056974bf4744a3cf557 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/84a8a86772a44056974bf4744a3cf557 2023-03-08 07:54:29,712 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/84a8a86772a44056974bf4744a3cf557, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:54:29,713 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 829ms, sequenceid=9, compaction requested=false 2023-03-08 07:54:29,713 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:29,713 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:29,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:29,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:29,715 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,715 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:29,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:29,716 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,716 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:29,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:29,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:29,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:29,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:29,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f198e7[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:29,717 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:29,717 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,722 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,723 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:29,723 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:29,723 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,723 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:29,723 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:29,724 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,724 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:29,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:29,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:29,724 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:29,724 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:29,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:29,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:29,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:29,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:29,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:29,727 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:29,727 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:29,727 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:29,727 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,727 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:29,728 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,728 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:29,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:29,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,728 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:29,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:29,729 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:29,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:29,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:29,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,730 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:29,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,732 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:29,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:29,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,733 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:29,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:29,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:29,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:29,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:29,736 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:29,736 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:29,836 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:29,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:29,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:29,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:29,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,842 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:29,842 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:29,842 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:29,842 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:29,843 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,843 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,845 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,845 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:29,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:29,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:29,846 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:29,846 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:29,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:29,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,847 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,847 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:29,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:29,847 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:29,847 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:29,847 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:30,260 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120199ab137e2d78435e8c61d80f218f0559 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120199ab137e2d78435e8c61d80f218f0559 2023-03-08 07:54:30,666 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/255d2685c2974195bbcf5f5970e8272d 2023-03-08 07:54:30,671 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/255d2685c2974195bbcf5f5970e8272d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/255d2685c2974195bbcf5f5970e8272d 2023-03-08 07:54:30,676 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/255d2685c2974195bbcf5f5970e8272d, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 07:54:30,677 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 830ms, sequenceid=13, compaction requested=true 2023-03-08 07:54:30,677 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:30,677 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:30,677 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:30,677 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:30,677 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,677 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:30,677 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:30,679 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:30,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:30,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:30,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60d71631[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:30,680 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:30,680 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,681 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,681 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:30,681 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:30,681 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:30,681 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,682 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:30,683 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,683 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:30,683 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:30,683 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:30,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:30,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:30,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:30,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:30,685 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:30,685 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:30,685 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:30,685 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,685 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:30,687 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,687 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:30,687 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,687 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:30,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,689 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,690 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:30,690 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:30,691 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:30,690 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:30,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:30,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:30,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:30,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,791 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:30,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:30,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:30,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:30,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,798 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:30,798 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:30,798 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:30,798 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:30,798 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,798 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,799 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,799 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:30,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:30,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:30,800 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:30,800 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:30,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:30,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:30,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:30,801 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:30,801 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:30,801 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:30,801 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:30,801 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:30,801 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:30,801 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:30,801 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:31,214 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c6910d52e81e49bc9ef84cdb6aa7c26e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c6910d52e81e49bc9ef84cdb6aa7c26e 2023-03-08 07:54:31,620 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/4e7b44e5dbd1442ba1533703ca65e988 2023-03-08 07:54:31,624 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/4e7b44e5dbd1442ba1533703ca65e988 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/4e7b44e5dbd1442ba1533703ca65e988 2023-03-08 07:54:31,628 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/4e7b44e5dbd1442ba1533703ca65e988, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 07:54:31,629 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 828ms, sequenceid=17, compaction requested=true 2023-03-08 07:54:31,629 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:31,629 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:31,629 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:31,629 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:31,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:31,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:31,631 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:31,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:31,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:31,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:31,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:31,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:31,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:31,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@172ae819[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:31,633 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:31,633 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,635 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,635 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:31,635 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:31,635 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:31,635 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,635 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:31,636 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:31,636 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,636 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:31,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:31,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:31,636 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:31,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:31,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:31,638 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:31,638 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:31,638 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:31,638 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,638 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:31,640 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,640 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:31,640 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:31,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:31,640 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:31,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,641 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:31,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:31,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:31,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:31,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,643 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:31,643 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:31,644 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,644 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,644 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:31,644 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:31,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:31,744 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:31,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:31,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:31,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:31,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,751 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:31,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:31,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:31,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:31,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,753 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,753 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:31,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:31,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:31,754 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:31,754 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:31,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:31,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:31,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:31,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:31,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:31,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:31,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:31,755 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:31,755 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:31,755 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:31,755 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:32,168 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284b30151c8b404a0ba4dd9fa8cfafb774 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284b30151c8b404a0ba4dd9fa8cfafb774 2023-03-08 07:54:32,427 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:54:32,574 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/76288e7c75bf49fcbb660139e426cd7f 2023-03-08 07:54:32,580 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/76288e7c75bf49fcbb660139e426cd7f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/76288e7c75bf49fcbb660139e426cd7f 2023-03-08 07:54:32,584 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/76288e7c75bf49fcbb660139e426cd7f, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 07:54:32,585 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 830ms, sequenceid=21, compaction requested=true 2023-03-08 07:54:32,585 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:32,585 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:32,589 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,589 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:32,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:32,589 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,589 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:32,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:32,591 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:32,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e675ccb[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:32,591 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,592 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,592 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,592 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:32,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:32,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:32,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:32,595 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:32,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:32,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:32,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:32,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:32,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:32,597 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:32,597 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:32,597 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:32,597 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,597 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:32,599 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,599 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:32,599 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:32,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:32,599 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:32,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,600 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,602 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,602 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,602 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:32,602 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:32,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:32,602 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:32,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:32,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:32,602 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:32,602 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,603 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:32,603 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:32,603 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,603 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:32,603 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:32,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:32,703 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:32,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:32,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:32,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:32,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,709 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:32,709 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:32,709 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:32,709 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:32,710 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,710 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,711 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:32,711 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:32,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:32,712 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:32,712 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:32,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:32,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,713 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:32,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:32,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:32,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:32,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:32,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:32,713 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:32,713 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:32,713 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:32,713 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:33,126 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123127c13c4a525e49cc9739336335e1b9cb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123127c13c4a525e49cc9739336335e1b9cb 2023-03-08 07:54:33,532 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/97f68702a7654f17ad577889f4f55a62 2023-03-08 07:54:33,537 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/97f68702a7654f17ad577889f4f55a62 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/97f68702a7654f17ad577889f4f55a62 2023-03-08 07:54:33,541 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/97f68702a7654f17ad577889f4f55a62, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 07:54:33,542 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 829ms, sequenceid=25, compaction requested=true 2023-03-08 07:54:33,542 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:33,542 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:33,543 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:33,543 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:33,543 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,543 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:33,543 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:33,546 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,546 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:33,546 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,546 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:33,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:33,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60dfc740[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:33,547 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:33,547 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,548 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,549 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:33,549 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:33,549 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:33,549 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,549 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:33,550 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,550 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:33,550 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:33,550 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:33,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:33,552 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:33,552 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:33,552 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:33,552 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,552 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:33,553 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,554 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:33,554 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:33,554 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,555 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:33,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:33,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:33,560 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,660 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:33,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:33,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:33,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:33,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,667 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:33,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:33,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:33,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:33,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,668 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,668 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:33,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:33,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:33,669 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:33,669 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:33,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:33,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:33,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:33,670 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:33,670 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:33,670 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:33,670 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:33,670 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:33,670 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:33,670 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:33,670 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:34,083 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019edf702c897748a68b6c2b696b1e40f7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601019edf702c897748a68b6c2b696b1e40f7 2023-03-08 07:54:34,489 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/6d65c147663c4bb08c507bd3d01ff630 2023-03-08 07:54:34,494 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/6d65c147663c4bb08c507bd3d01ff630 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/6d65c147663c4bb08c507bd3d01ff630 2023-03-08 07:54:34,497 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/6d65c147663c4bb08c507bd3d01ff630, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 07:54:34,498 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 828ms, sequenceid=29, compaction requested=true 2023-03-08 07:54:34,498 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:34,498 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:34,499 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:34,499 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:34,499 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,499 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:34,499 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:34,500 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,500 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:34,501 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,501 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:34,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:34,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@386123f4[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:34,502 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:34,502 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,504 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:34,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:34,504 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,504 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:34,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:34,506 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,506 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:34,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:34,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:34,506 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:34,507 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:34,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:34,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:34,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:34,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:34,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:34,509 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:34,509 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:34,509 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:34,509 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,509 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:34,510 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,510 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:34,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:34,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,510 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:34,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:34,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:34,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,513 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:34,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:34,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,515 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:34,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,516 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:34,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:34,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:34,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:34,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,517 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:34,517 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:34,617 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:34,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:34,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:34,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,624 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:34,624 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:34,624 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:34,624 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:34,624 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,625 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,629 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,629 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:34,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:34,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:34,630 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:34,630 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:34,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:34,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:34,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:34,631 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:34,631 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:34,631 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:35,045 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037e15707949ef46b3a4498c51ba1b68eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037e15707949ef46b3a4498c51ba1b68eb 2023-03-08 07:54:35,450 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/0dd9d1a8be0840b791de186f814ec6a9 2023-03-08 07:54:35,455 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/0dd9d1a8be0840b791de186f814ec6a9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0dd9d1a8be0840b791de186f814ec6a9 2023-03-08 07:54:35,459 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0dd9d1a8be0840b791de186f814ec6a9, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 07:54:35,460 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 829ms, sequenceid=33, compaction requested=true 2023-03-08 07:54:35,460 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:35,460 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:35,462 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,462 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:35,462 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,462 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78cb6ec1[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:35,463 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:35,464 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,464 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,465 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:35,465 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:35,465 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:35,465 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,465 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:35,466 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,466 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,466 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:35,466 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:35,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:35,468 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:35,468 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:35,468 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:35,468 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,468 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:35,471 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,471 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,471 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:35,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:35,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:35,471 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:35,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,472 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:35,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,475 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,475 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:35,475 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:35,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:35,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:35,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:35,475 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:35,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:35,475 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,481 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,581 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:35,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:35,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:35,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:35,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,588 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:35,588 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:35,588 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:35,588 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:35,589 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,589 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,590 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,590 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:35,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:35,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:35,590 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:35,590 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:35,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:35,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:35,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:35,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:35,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:35,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:35,592 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:35,592 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:35,592 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:35,592 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:36,005 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090719e9a3fccda64f53babef9944879c2b3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090719e9a3fccda64f53babef9944879c2b3 2023-03-08 07:54:36,011 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/0114ee5e71cc4321a280e659c6e1ca43 2023-03-08 07:54:36,017 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/0114ee5e71cc4321a280e659c6e1ca43 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0114ee5e71cc4321a280e659c6e1ca43 2023-03-08 07:54:36,021 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0114ee5e71cc4321a280e659c6e1ca43, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 07:54:36,022 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 430ms, sequenceid=37, compaction requested=true 2023-03-08 07:54:36,023 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:36,023 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:36,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:36,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:36,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:36,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:36,024 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,024 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:36,025 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,025 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:36,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:36,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c1ef417[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:36,026 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:36,026 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,028 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:36,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:36,028 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,028 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:36,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:36,029 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,029 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:36,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:36,030 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:36,030 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:36,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:36,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:36,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:36,032 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:36,032 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:36,032 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:36,032 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,032 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:36,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:36,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:36,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:36,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:36,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:36,035 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:36,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:36,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,036 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:36,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,038 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,038 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,038 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:36,038 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:36,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:36,038 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:36,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:36,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:36,038 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:36,038 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,039 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,039 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,039 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:36,039 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:36,039 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:36,139 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:36,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:36,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:36,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:36,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,146 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:36,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:36,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 07:54:36,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 07:54:36,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,148 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,148 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:36,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:36,149 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,149 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 07:54:36,150 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 07:54:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 07:54:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:36,150 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:36,150 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. started... 2023-03-08 07:54:36,150 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e906f28e8a2d2e81f61b506d78f3c6d3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:36,563 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112007dbbbb515a048dbbcbca6f61eebd701 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112007dbbbb515a048dbbcbca6f61eebd701 2023-03-08 07:54:36,969 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/8d60d651753443fab21ba8b0cf0b859e 2023-03-08 07:54:36,973 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/8d60d651753443fab21ba8b0cf0b859e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/8d60d651753443fab21ba8b0cf0b859e 2023-03-08 07:54:36,977 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/8d60d651753443fab21ba8b0cf0b859e, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 07:54:36,978 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e906f28e8a2d2e81f61b506d78f3c6d3 in 828ms, sequenceid=41, compaction requested=true 2023-03-08 07:54:36,978 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:36,978 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:36,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:36,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:36,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 07:54:36,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 07:54:36,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,980 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:36,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:36,980 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,980 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:36,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 07:54:36,982 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 07:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32e5c313[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:36,982 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,983 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,983 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 07:54:36,983 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 07:54:36,983 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:36,983 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,983 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:36,984 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,984 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 07:54:36,984 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:36,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:36,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:36,984 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 07:54:36,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:36,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:36,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:36,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:36,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:54:36,987 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 07:54:36,987 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:36,987 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:36,987 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,987 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:36,989 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,989 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:36,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:36,989 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:36,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:36,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:36,990 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,991 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:36,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 07:54:36,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 07:54:36,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:36,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:36,995 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:36,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:36,995 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:36,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:36,996 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:37,096 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:37,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:37,098 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-08 07:54:37,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:37,100 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:37,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:37,101 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262677101 2023-03-08 07:54:37,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:54:37,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 07:54:37,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:37,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x21a6eaa4 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:37,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@334ee571, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:37,109 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:37,949 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d4d1391270164c13a5ea61ba3c77c9d9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103d4d1391270164c13a5ea61ba3c77c9d9 2023-03-08 07:54:37,950 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:37,964 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:54:37,965 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40378, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:54:37,972 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:54:37,973 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45844, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:54:37,982 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5507ea894a6b486b9dd2d61af50f9527 first=Optional[r3] last=Optional[r6] 2023-03-08 07:54:37,983 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5507ea894a6b486b9dd2d61af50f9527}] 2023-03-08 07:54:37,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5507ea894a6b486b9dd2d61af50f9527 for inclusion in e906f28e8a2d2e81f61b506d78f3c6d3/family1 2023-03-08 07:54:37,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 07:54:37,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:37,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:37,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5507ea894a6b486b9dd2d61af50f9527 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tbth066hs55o0qm4kfkjn0k6dkgba8gv0p0k9ed0bn2p4etb5fg30919jfgis96c/family1/5507ea894a6b486b9dd2d61af50f9527 2023-03-08 07:54:37,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tbth066hs55o0qm4kfkjn0k6dkgba8gv0p0k9ed0bn2p4etb5fg30919jfgis96c/family1/5507ea894a6b486b9dd2d61af50f9527 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_ 2023-03-08 07:54:37,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tbth066hs55o0qm4kfkjn0k6dkgba8gv0p0k9ed0bn2p4etb5fg30919jfgis96c/family1/5507ea894a6b486b9dd2d61af50f9527 into e906f28e8a2d2e81f61b506d78f3c6d3/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_ - updating store file list. 2023-03-08 07:54:37,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_ into e906f28e8a2d2e81f61b506d78f3c6d3/family1 2023-03-08 07:54:37,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tbth066hs55o0qm4kfkjn0k6dkgba8gv0p0k9ed0bn2p4etb5fg30919jfgis96c/family1/5507ea894a6b486b9dd2d61af50f9527 into e906f28e8a2d2e81f61b506d78f3c6d3/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_) 2023-03-08 07:54:37,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tbth066hs55o0qm4kfkjn0k6dkgba8gv0p0k9ed0bn2p4etb5fg30919jfgis96c/family1/5507ea894a6b486b9dd2d61af50f9527 2023-03-08 07:54:37,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:54:38,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:54:38,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): e906f28e8a2d2e81f61b506d78f3c6d3/family1 is initiating minor compaction 2023-03-08 07:54:38,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@158b69ee; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:38,002 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e906f28e8a2d2e81f61b506d78f3c6d3/family1 in testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:54:38,002 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/84a8a86772a44056974bf4744a3cf557, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/255d2685c2974195bbcf5f5970e8272d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/4e7b44e5dbd1442ba1533703ca65e988, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/76288e7c75bf49fcbb660139e426cd7f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/97f68702a7654f17ad577889f4f55a62, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/6d65c147663c4bb08c507bd3d01ff630, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0dd9d1a8be0840b791de186f814ec6a9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0114ee5e71cc4321a280e659c6e1ca43, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/8d60d651753443fab21ba8b0cf0b859e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp, totalSize=50.0 K 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 84a8a86772a44056974bf4744a3cf557, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 255d2685c2974195bbcf5f5970e8272d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 4e7b44e5dbd1442ba1533703ca65e988, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 76288e7c75bf49fcbb660139e426cd7f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 97f68702a7654f17ad577889f4f55a62, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 6d65c147663c4bb08c507bd3d01ff630, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 0dd9d1a8be0840b791de186f814ec6a9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 0114ee5e71cc4321a280e659c6e1ca43, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 8d60d651753443fab21ba8b0cf0b859e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:54:38,002 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:54:38,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:38,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:38,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284b30151c8b404a0ba4dd9fa8cfafb774 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284b30151c8b404a0ba4dd9fa8cfafb774 2023-03-08 07:54:38,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123127c13c4a525e49cc9739336335e1b9cb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123127c13c4a525e49cc9739336335e1b9cb 2023-03-08 07:54:38,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601019edf702c897748a68b6c2b696b1e40f7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601019edf702c897748a68b6c2b696b1e40f7 2023-03-08 07:54:38,011 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037e15707949ef46b3a4498c51ba1b68eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037e15707949ef46b3a4498c51ba1b68eb 2023-03-08 07:54:38,011 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 07:54:38,011 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,062 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:54:38,062 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x60424cf6 to 127.0.0.1:52387 2023-03-08 07:54:38,063 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:54:38,066 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:54:38,066 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33de556b to 127.0.0.1:52387 2023-03-08 07:54:38,066 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:54:38,080 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): e906f28e8a2d2e81f61b506d78f3c6d3#family1#compaction#49 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 76.92 MB/second 2023-03-08 07:54:38,105 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051f009e480e784de49d4e32731c227fbf to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512051f009e480e784de49d4e32731c227fbf 2023-03-08 07:54:38,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:38,132 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be94e729e49347b29d81d534184001e4 first=Optional[r00] last=Optional[r2] 2023-03-08 07:54:38,132 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be94e729e49347b29d81d534184001e4}] 2023-03-08 07:54:38,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be94e729e49347b29d81d534184001e4 for inclusion in e906f28e8a2d2e81f61b506d78f3c6d3/family1 2023-03-08 07:54:38,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 07:54:38,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:38,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:38,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be94e729e49347b29d81d534184001e4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fogo6lnjog79loo23kfe4uq50a0u9in7jn5hrqlf2rr4df3g1c0b1tg3hrnt8ef3/family1/be94e729e49347b29d81d534184001e4 2023-03-08 07:54:38,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fogo6lnjog79loo23kfe4uq50a0u9in7jn5hrqlf2rr4df3g1c0b1tg3hrnt8ef3/family1/be94e729e49347b29d81d534184001e4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/1f74dda0c1504f409bdf8f2cec12d6ea_SeqId_46_ 2023-03-08 07:54:38,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fogo6lnjog79loo23kfe4uq50a0u9in7jn5hrqlf2rr4df3g1c0b1tg3hrnt8ef3/family1/be94e729e49347b29d81d534184001e4 into e906f28e8a2d2e81f61b506d78f3c6d3/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/1f74dda0c1504f409bdf8f2cec12d6ea_SeqId_46_ - updating store file list. 2023-03-08 07:54:38,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/1f74dda0c1504f409bdf8f2cec12d6ea_SeqId_46_ into e906f28e8a2d2e81f61b506d78f3c6d3/family1 2023-03-08 07:54:38,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fogo6lnjog79loo23kfe4uq50a0u9in7jn5hrqlf2rr4df3g1c0b1tg3hrnt8ef3/family1/be94e729e49347b29d81d534184001e4 into e906f28e8a2d2e81f61b506d78f3c6d3/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/1f74dda0c1504f409bdf8f2cec12d6ea_SeqId_46_) 2023-03-08 07:54:38,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fogo6lnjog79loo23kfe4uq50a0u9in7jn5hrqlf2rr4df3g1c0b1tg3hrnt8ef3/family1/be94e729e49347b29d81d534184001e4 2023-03-08 07:54:38,146 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:38,147 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:38,148 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511308859e851d4e94229a2d6d651b230b323 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511308859e851d4e94229a2d6d651b230b323 2023-03-08 07:54:38,149 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120199ab137e2d78435e8c61d80f218f0559 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120199ab137e2d78435e8c61d80f218f0559 2023-03-08 07:54:38,150 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c6910d52e81e49bc9ef84cdb6aa7c26e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c6910d52e81e49bc9ef84cdb6aa7c26e 2023-03-08 07:54:38,150 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-08 07:54:38,150 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 07:54:38,151 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 07:54:38,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 07:54:38,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:38,152 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:54:38,153 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0530 sec 2023-03-08 07:54:38,264 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=255 (was 252) - Thread LEAK? -, OpenFileDescriptor=618 (was 606) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 278), ProcessCount=170 (was 170), AvailableMemoryMB=7589 (was 7602) 2023-03-08 07:54:38,280 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=255, OpenFileDescriptor=614, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=170, AvailableMemoryMB=7588 2023-03-08 07:54:38,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 07:54:38,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:38,284 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:54:38,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-08 07:54:38,285 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:54:38,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 07:54:38,287 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:38,287 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 empty. 2023-03-08 07:54:38,288 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:38,288 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-08 07:54:38,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 07:54:38,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 07:54:38,696 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 07:54:38,697 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 88f4fcc1aaf1e4b2430d092d92ce36a8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:38,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 07:54:38,908 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/.tmp/family1/d2ab7fb65c444c89b1910a285ffd546c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/d2ab7fb65c444c89b1910a285ffd546c 2023-03-08 07:54:38,912 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e906f28e8a2d2e81f61b506d78f3c6d3/family1 of e906f28e8a2d2e81f61b506d78f3c6d3 into d2ab7fb65c444c89b1910a285ffd546c(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:54:38,912 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:54:38,913 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678262078001; duration=0sec 2023-03-08 07:54:38,913 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:38,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/84a8a86772a44056974bf4744a3cf557, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/255d2685c2974195bbcf5f5970e8272d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/4e7b44e5dbd1442ba1533703ca65e988, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/76288e7c75bf49fcbb660139e426cd7f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/97f68702a7654f17ad577889f4f55a62, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/6d65c147663c4bb08c507bd3d01ff630, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0dd9d1a8be0840b791de186f814ec6a9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0114ee5e71cc4321a280e659c6e1ca43, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/8d60d651753443fab21ba8b0cf0b859e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_] to archive 2023-03-08 07:54:38,932 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:38,934 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/84a8a86772a44056974bf4744a3cf557 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/84a8a86772a44056974bf4744a3cf557 2023-03-08 07:54:38,935 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/255d2685c2974195bbcf5f5970e8272d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/255d2685c2974195bbcf5f5970e8272d 2023-03-08 07:54:38,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/4e7b44e5dbd1442ba1533703ca65e988 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/4e7b44e5dbd1442ba1533703ca65e988 2023-03-08 07:54:38,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/76288e7c75bf49fcbb660139e426cd7f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/76288e7c75bf49fcbb660139e426cd7f 2023-03-08 07:54:38,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/97f68702a7654f17ad577889f4f55a62 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/97f68702a7654f17ad577889f4f55a62 2023-03-08 07:54:38,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/6d65c147663c4bb08c507bd3d01ff630 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/6d65c147663c4bb08c507bd3d01ff630 2023-03-08 07:54:38,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0dd9d1a8be0840b791de186f814ec6a9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0dd9d1a8be0840b791de186f814ec6a9 2023-03-08 07:54:38,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0114ee5e71cc4321a280e659c6e1ca43 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/0114ee5e71cc4321a280e659c6e1ca43 2023-03-08 07:54:38,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/8d60d651753443fab21ba8b0cf0b859e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/8d60d651753443fab21ba8b0cf0b859e 2023-03-08 07:54:38,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/family1/f3d651cd5e94487c8d51f02f4099b7c6_SeqId_44_ 2023-03-08 07:54:39,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:39,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 88f4fcc1aaf1e4b2430d092d92ce36a8, disabling compactions & flushes 2023-03-08 07:54:39,103 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. after waiting 0 ms 2023-03-08 07:54:39,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,103 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:39,107 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:54:39,108 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262079107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262079107"}]},"ts":"1678262079107"} 2023-03-08 07:54:39,109 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:54:39,110 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:54:39,110 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262079110"}]},"ts":"1678262079110"} 2023-03-08 07:54:39,111 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-08 07:54:39,115 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, ASSIGN}] 2023-03-08 07:54:39,117 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, ASSIGN 2023-03-08 07:54:39,118 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:39,269 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:39,269 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262079269"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262079269"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262079269"}]},"ts":"1678262079269"} 2023-03-08 07:54:39,271 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:39,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 07:54:39,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 88f4fcc1aaf1e4b2430d092d92ce36a8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:54:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,429 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,430 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:39,430 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:39,431 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88f4fcc1aaf1e4b2430d092d92ce36a8 columnFamilyName family1 2023-03-08 07:54:39,432 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(310): Store=88f4fcc1aaf1e4b2430d092d92ce36a8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:39,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:39,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:39,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 88f4fcc1aaf1e4b2430d092d92ce36a8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67736285, jitterRate=0.00934930145740509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:54:39,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:39,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., pid=41, masterSystemTime=1678262079423 2023-03-08 07:54:39,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:39,441 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:39,441 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262079441"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262079441"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262079441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262079441"}]},"ts":"1678262079441"} 2023-03-08 07:54:39,444 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-08 07:54:39,444 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 171 msec 2023-03-08 07:54:39,445 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-08 07:54:39,445 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, ASSIGN in 329 msec 2023-03-08 07:54:39,446 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:54:39,446 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262079446"}]},"ts":"1678262079446"} 2023-03-08 07:54:39,447 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-08 07:54:39,449 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:54:39,450 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1670 sec 2023-03-08 07:54:40,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 07:54:40,391 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-08 07:54:40,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:40,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,417 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:40,418 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:40,418 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:40,418 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:40,418 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,418 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,419 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:40,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:40,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:40,419 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:40,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,420 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:40,420 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:40,420 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:40,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,421 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,421 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:40,421 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:40,421 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:40,421 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:40,421 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:40,421 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:40,421 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:40,421 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:40,422 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:54:40,844 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130dde24e25c9f54f6b9da30d46a6851b3c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130dde24e25c9f54f6b9da30d46a6851b3c 2023-03-08 07:54:41,248 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/b5042a4b041b4e7ea9500d4050061cc0 2023-03-08 07:54:41,253 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/b5042a4b041b4e7ea9500d4050061cc0 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b5042a4b041b4e7ea9500d4050061cc0 2023-03-08 07:54:41,257 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b5042a4b041b4e7ea9500d4050061cc0, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-08 07:54:41,258 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 836ms, sequenceid=5, compaction requested=false 2023-03-08 07:54:41,258 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:41,258 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:41,258 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:41,258 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:41,259 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:41,259 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,259 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:41,259 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:41,260 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,260 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:41,260 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,260 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:41,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3aa2838[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:41,262 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:41,262 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,263 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:41,263 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:41,263 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:41,263 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,263 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:41,265 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,265 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:41,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:41,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:41,265 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:41,265 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:41,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:41,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:41,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:41,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:41,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:41,267 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:41,267 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:41,267 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:41,267 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,267 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:41,269 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,269 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,269 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:41,269 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:41,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:41,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,270 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:41,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:41,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:41,272 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,272 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:41,273 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,272 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:41,273 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:41,273 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:41,273 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:41,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:41,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:41,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:41,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:41,373 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:41,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:41,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:41,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:41,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,380 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:41,380 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:41,380 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:41,380 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:41,381 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,381 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,382 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,382 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:41,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:41,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:41,382 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:41,382 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:41,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:41,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,383 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,383 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:41,383 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:41,383 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:41,383 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:41,383 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:41,383 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:41,384 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:41,384 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:41,384 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:41,384 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:54:41,798 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284764b15b415f47c8b07cf0beaddbfda0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284764b15b415f47c8b07cf0beaddbfda0 2023-03-08 07:54:42,205 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/769f7f31416c458db35dde7d9b7839ee 2023-03-08 07:54:42,211 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/769f7f31416c458db35dde7d9b7839ee as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/769f7f31416c458db35dde7d9b7839ee 2023-03-08 07:54:42,216 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/769f7f31416c458db35dde7d9b7839ee, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:54:42,217 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 833ms, sequenceid=9, compaction requested=false 2023-03-08 07:54:42,217 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:42,217 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:42,217 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:42,217 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:42,217 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,217 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:42,217 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:42,220 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:42,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:42,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:42,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:42,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:42,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:42,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20b4e3ea[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:42,222 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:42,222 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,223 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:42,224 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,224 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:42,225 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:42,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:42,228 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:42,228 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:42,228 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:42,228 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,228 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:42,229 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,229 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,229 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:42,230 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:42,231 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:42,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,236 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,236 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,236 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,236 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:42,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:42,237 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:42,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:42,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:42,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:42,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:42,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:42,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 07:54:42,337 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:42,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:42,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:42,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:42,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,345 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:42,345 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:42,345 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:42,345 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:42,345 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,345 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,346 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:42,346 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:42,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:42,347 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:42,347 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:42,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:42,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,348 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:42,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:42,348 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:42,348 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:42,348 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:42,348 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:42,348 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:42,348 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:42,348 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:42,348 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:42,765 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f5b1752137774e90b327e63aa48dfd7f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201f5b1752137774e90b327e63aa48dfd7f 2023-03-08 07:54:43,172 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/9df9c0c797f745be80ad313a0a478ed5 2023-03-08 07:54:43,178 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/9df9c0c797f745be80ad313a0a478ed5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/9df9c0c797f745be80ad313a0a478ed5 2023-03-08 07:54:43,183 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/9df9c0c797f745be80ad313a0a478ed5, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 07:54:43,184 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 836ms, sequenceid=13, compaction requested=true 2023-03-08 07:54:43,184 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:43,184 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:43,185 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,185 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:43,186 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:43,186 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:43,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:43,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b5115a7[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:43,187 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:43,187 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,188 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,188 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:43,188 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:43,188 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:43,189 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,189 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:43,191 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,191 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:43,191 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:43,191 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:43,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:43,193 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:43,193 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:43,193 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:43,193 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,193 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:43,195 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,195 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:43,195 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:43,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:43,195 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,196 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:43,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,198 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:43,198 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,199 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:43,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:43,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:43,199 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:43,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:43,299 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:43,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:43,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:43,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:43,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,306 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:43,306 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:43,306 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:43,306 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:43,306 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,306 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,308 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:43,308 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:43,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:43,308 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:43,308 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:43,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:43,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:43,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:43,309 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:43,309 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:43,309 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:43,309 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:43,309 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:43,309 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:43,310 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:43,310 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:43,721 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205df20c73486e34a7dad426692dfeb279b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205df20c73486e34a7dad426692dfeb279b 2023-03-08 07:54:44,127 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/c5eb397720a14bac94b218cb6f830ee2 2023-03-08 07:54:44,132 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/c5eb397720a14bac94b218cb6f830ee2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/c5eb397720a14bac94b218cb6f830ee2 2023-03-08 07:54:44,136 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/c5eb397720a14bac94b218cb6f830ee2, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 07:54:44,137 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 827ms, sequenceid=17, compaction requested=true 2023-03-08 07:54:44,137 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:44,137 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:44,139 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,139 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:44,140 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:44,140 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:44,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:44,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6730bc85[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:44,141 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:44,141 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,142 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:44,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:44,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:44,142 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,142 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:44,144 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:44,144 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,144 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:44,144 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:44,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:44,146 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:44,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:44,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:44,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,146 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:44,147 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,147 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:44,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,147 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:44,148 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,149 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:44,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:44,152 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:44,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,152 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:44,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:44,252 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:44,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:44,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:44,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:44,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,259 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:44,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:44,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:44,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:44,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,260 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,260 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:44,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:44,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:44,261 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:44,261 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:44,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:44,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:44,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:44,262 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:44,262 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:44,262 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:44,262 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:44,262 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:44,262 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:44,262 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:44,262 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:44,678 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512281bddc08734464fcab35b7f3e605d56a3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512281bddc08734464fcab35b7f3e605d56a3 2023-03-08 07:54:44,964 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:54:45,083 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/f9fc960b45b842fc85ced1a1619d382c 2023-03-08 07:54:45,088 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/f9fc960b45b842fc85ced1a1619d382c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/f9fc960b45b842fc85ced1a1619d382c 2023-03-08 07:54:45,092 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/f9fc960b45b842fc85ced1a1619d382c, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 07:54:45,093 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 831ms, sequenceid=21, compaction requested=true 2023-03-08 07:54:45,093 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:45,093 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:45,097 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,097 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:45,098 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,098 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:45,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:45,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3509a3b3[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:45,099 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:45,099 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,100 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,100 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:45,100 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:45,100 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,100 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:45,100 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:45,101 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:45,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:45,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:45,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:45,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:45,103 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:45,103 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:45,103 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:45,103 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,103 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:45,106 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,106 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:45,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:45,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,106 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:45,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:45,107 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:45,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,108 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:45,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:45,110 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:45,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:45,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:45,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:45,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:45,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:45,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,111 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,111 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:45,111 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:45,211 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:45,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:45,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:45,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:45,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,218 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:45,218 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:45,218 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:45,218 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:45,220 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,220 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,221 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,221 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:45,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:45,221 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:45,221 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,222 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:45,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,222 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:45,222 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:45,222 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:45,222 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:45,222 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:45,222 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:45,222 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:45,223 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:45,223 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:45,223 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:45,634 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123106d24e066d474c45babc1bbbde0091e7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123106d24e066d474c45babc1bbbde0091e7 2023-03-08 07:54:46,040 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/583ae279f3bc4d708585d76879378bdc 2023-03-08 07:54:46,045 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/583ae279f3bc4d708585d76879378bdc as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/583ae279f3bc4d708585d76879378bdc 2023-03-08 07:54:46,049 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/583ae279f3bc4d708585d76879378bdc, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 07:54:46,050 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 827ms, sequenceid=25, compaction requested=true 2023-03-08 07:54:46,050 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:46,050 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:46,052 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:46,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:46,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3225bd4b[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:46,054 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:46,054 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,055 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,055 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:46,055 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:46,055 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:46,055 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,055 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:46,056 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,056 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:46,056 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:46,056 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:46,058 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:46,058 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:46,058 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:46,058 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,058 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:46,061 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,061 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:46,061 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:46,061 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,062 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:46,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,064 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,064 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,064 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:46,065 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:46,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 07:54:46,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:46,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:46,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:46,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:46,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:46,065 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:46,165 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:46,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:46,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:46,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:46,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,172 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:46,172 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:46,172 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:46,172 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:46,172 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,172 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,173 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,173 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:46,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:46,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:46,174 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,174 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:46,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:46,175 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:46,175 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:46,175 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:46,589 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101db0f35ccbc9943ad927642a8f7bdf0a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101db0f35ccbc9943ad927642a8f7bdf0a2 2023-03-08 07:54:46,595 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/d0708b62cfab4b91af1cab8061857c28 2023-03-08 07:54:46,599 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/d0708b62cfab4b91af1cab8061857c28 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/d0708b62cfab4b91af1cab8061857c28 2023-03-08 07:54:46,603 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/d0708b62cfab4b91af1cab8061857c28, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 07:54:46,603 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 428ms, sequenceid=29, compaction requested=true 2023-03-08 07:54:46,603 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:46,603 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:46,604 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:46,604 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:46,604 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,604 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:46,604 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:46,605 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,605 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:46,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:46,605 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,606 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:46,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:46,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:46,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:46,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:46,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@412a7a85[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:46,607 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:46,607 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,609 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,609 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:46,609 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,609 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:46,609 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:46,609 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:46,610 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,610 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:46,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:46,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:46,610 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:46,611 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:46,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:46,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:46,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:46,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:46,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:46,612 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:46,612 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:46,612 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:46,613 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,613 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:46,617 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,617 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:46,617 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,617 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:46,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,618 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:46,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,620 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:46,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:46,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:46,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:46,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:46,620 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,621 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,621 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:46,621 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:46,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:46,721 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:46,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:46,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:46,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:46,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,727 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:46,727 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:46,727 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:46,727 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:46,728 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,728 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,729 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,729 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:46,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:46,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:46,729 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:46,729 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:46,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,730 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,730 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:46,730 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:46,730 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:46,730 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:46,730 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:46,730 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:46,730 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:46,730 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:46,730 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:47,144 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ab79de1293e54c30852230014157a05d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103ab79de1293e54c30852230014157a05d 2023-03-08 07:54:47,551 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/71dfeb61cb5c4165939f49defb2b3a18 2023-03-08 07:54:47,555 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/71dfeb61cb5c4165939f49defb2b3a18 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/71dfeb61cb5c4165939f49defb2b3a18 2023-03-08 07:54:47,559 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/71dfeb61cb5c4165939f49defb2b3a18, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 07:54:47,559 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 829ms, sequenceid=33, compaction requested=true 2023-03-08 07:54:47,559 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:47,560 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:47,560 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:47,560 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:47,560 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,560 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:47,560 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:47,562 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,562 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:47,562 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,562 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@239c32b1[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:47,563 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:47,563 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,564 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,565 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,565 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:47,566 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:47,566 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:47,566 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,566 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:47,568 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:47,568 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:47,568 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:47,568 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,568 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:47,570 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,570 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:47,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,570 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:47,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:47,570 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:47,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,571 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:47,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:47,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:47,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:47,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:47,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:47,675 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:47,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:47,679 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:47,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:47,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,681 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:47,681 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:47,681 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:47,681 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:47,682 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,682 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,683 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:47,683 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:47,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:47,683 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:47,683 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:47,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,684 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:47,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:47,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:47,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:47,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:47,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:47,684 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:47,685 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:47,685 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:47,685 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:47,697 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907452138c9fa434eadaebbea2f45c69991 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907452138c9fa434eadaebbea2f45c69991 2023-03-08 07:54:48,102 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/118131eaefaa44419aeb9dd9617d0e53 2023-03-08 07:54:48,107 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/118131eaefaa44419aeb9dd9617d0e53 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/118131eaefaa44419aeb9dd9617d0e53 2023-03-08 07:54:48,111 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/118131eaefaa44419aeb9dd9617d0e53, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 07:54:48,112 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 427ms, sequenceid=37, compaction requested=true 2023-03-08 07:54:48,112 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:48,112 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:48,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:48,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:48,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:48,112 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:48,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,114 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:48,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:48,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:48,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bf0194a[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:48,115 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:48,115 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,116 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,117 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:48,117 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:48,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:48,117 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:48,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:48,121 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:48,121 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:48,121 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:48,121 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,121 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:48,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,123 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:48,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:48,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:48,123 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:48,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:48,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:48,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:48,124 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,129 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,129 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:48,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:48,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:48,130 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:48,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 07:54:48,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:48,130 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:48,130 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:48,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:48,230 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:48,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:48,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:48,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:48,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,237 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:48,237 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:48,237 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 07:54:48,237 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 07:54:48,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,239 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,239 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:48,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:48,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:48,239 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:48,239 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:48,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,240 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 07:54:48,240 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 07:54:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 07:54:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:48,241 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:48,241 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:48,241 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. started... 2023-03-08 07:54:48,241 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88f4fcc1aaf1e4b2430d092d92ce36a8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:48,652 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112068dcaaa54c474545b4ab082913e30060 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112068dcaaa54c474545b4ab082913e30060 2023-03-08 07:54:49,058 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/b3b5f00378104cfa936cae1a133d1fa6 2023-03-08 07:54:49,062 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/b3b5f00378104cfa936cae1a133d1fa6 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b3b5f00378104cfa936cae1a133d1fa6 2023-03-08 07:54:49,066 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b3b5f00378104cfa936cae1a133d1fa6, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 07:54:49,066 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 88f4fcc1aaf1e4b2430d092d92ce36a8 in 825ms, sequenceid=41, compaction requested=true 2023-03-08 07:54:49,066 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:49,066 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:49,066 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:49,067 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:49,067 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,067 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 07:54:49,067 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 07:54:49,068 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:49,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:49,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 07:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44a0389a[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:49,069 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 07:54:49,069 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,072 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 07:54:49,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 07:54:49,072 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:49,072 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,072 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:49,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 07:54:49,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:49,073 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 07:54:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 07:54:49,075 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 07:54:49,075 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:49,075 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:49,075 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,075 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:49,077 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,077 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,077 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:49,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:49,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:49,078 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,079 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:49,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:49,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 07:54:49,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:49,081 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:49,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 07:54:49,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:49,081 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:49,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:49,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:49,182 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:49,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:49,184 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:49,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:49,186 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:49,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:49,187 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262689187 2023-03-08 07:54:49,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:54:49,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 07:54:49,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:49,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x68bc28e2 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:49,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60a414f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:49,196 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,068 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010323cd858c5a6a4ca1b643c2c8c377045c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010323cd858c5a6a4ca1b643c2c8c377045c 2023-03-08 07:54:50,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:50,082 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:54:50,083 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34830, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:54:50,090 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:54:50,092 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58448, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:54:50,100 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b619665144e448d798507986f057909d first=Optional[r3] last=Optional[r6] 2023-03-08 07:54:50,101 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b619665144e448d798507986f057909d}] 2023-03-08 07:54:50,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b619665144e448d798507986f057909d for inclusion in 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:50,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 07:54:50,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:50,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:50,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b619665144e448d798507986f057909d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qmv1snkbveqaqegeo4m2n20e5oa9286nlhjbfrvk8slku8inn6vmkut68hmjcv51/family1/b619665144e448d798507986f057909d 2023-03-08 07:54:50,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qmv1snkbveqaqegeo4m2n20e5oa9286nlhjbfrvk8slku8inn6vmkut68hmjcv51/family1/b619665144e448d798507986f057909d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_ 2023-03-08 07:54:50,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qmv1snkbveqaqegeo4m2n20e5oa9286nlhjbfrvk8slku8inn6vmkut68hmjcv51/family1/b619665144e448d798507986f057909d into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_ - updating store file list. 2023-03-08 07:54:50,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_ into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:50,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qmv1snkbveqaqegeo4m2n20e5oa9286nlhjbfrvk8slku8inn6vmkut68hmjcv51/family1/b619665144e448d798507986f057909d into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_) 2023-03-08 07:54:50,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qmv1snkbveqaqegeo4m2n20e5oa9286nlhjbfrvk8slku8inn6vmkut68hmjcv51/family1/b619665144e448d798507986f057909d 2023-03-08 07:54:50,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:54:50,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:54:50,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 is initiating minor compaction 2023-03-08 07:54:50,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@45a97825; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:50,121 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:50,121 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/769f7f31416c458db35dde7d9b7839ee, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/9df9c0c797f745be80ad313a0a478ed5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/c5eb397720a14bac94b218cb6f830ee2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/f9fc960b45b842fc85ced1a1619d382c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/583ae279f3bc4d708585d76879378bdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/d0708b62cfab4b91af1cab8061857c28, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/71dfeb61cb5c4165939f49defb2b3a18, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/118131eaefaa44419aeb9dd9617d0e53, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b3b5f00378104cfa936cae1a133d1fa6, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp, totalSize=50.4 K 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 769f7f31416c458db35dde7d9b7839ee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 9df9c0c797f745be80ad313a0a478ed5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting c5eb397720a14bac94b218cb6f830ee2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting f9fc960b45b842fc85ced1a1619d382c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 583ae279f3bc4d708585d76879378bdc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting d0708b62cfab4b91af1cab8061857c28, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 71dfeb61cb5c4165939f49defb2b3a18, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 118131eaefaa44419aeb9dd9617d0e53, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting b3b5f00378104cfa936cae1a133d1fa6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:54:50,121 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:54:50,123 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:50,124 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:50,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512281bddc08734464fcab35b7f3e605d56a3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512281bddc08734464fcab35b7f3e605d56a3 2023-03-08 07:54:50,127 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123106d24e066d474c45babc1bbbde0091e7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123106d24e066d474c45babc1bbbde0091e7 2023-03-08 07:54:50,128 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101db0f35ccbc9943ad927642a8f7bdf0a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101db0f35ccbc9943ad927642a8f7bdf0a2 2023-03-08 07:54:50,130 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103ab79de1293e54c30852230014157a05d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103ab79de1293e54c30852230014157a05d 2023-03-08 07:54:50,130 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-08 07:54:50,130 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:50,148 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 88f4fcc1aaf1e4b2430d092d92ce36a8#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 76.92 MB/second 2023-03-08 07:54:50,967 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/df99e25d9ba84580a338fa7aab378b40 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/df99e25d9ba84580a338fa7aab378b40 2023-03-08 07:54:50,972 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 of 88f4fcc1aaf1e4b2430d092d92ce36a8 into df99e25d9ba84580a338fa7aab378b40(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:54:50,972 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:50,972 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678262090121; duration=0sec 2023-03-08 07:54:50,972 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:50,980 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056195188e60b5401b9fae3d9895b6ba24 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512056195188e60b5401b9fae3d9895b6ba24 2023-03-08 07:54:50,981 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:51,006 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ce2d248c97134ef7880429885599b03b first=Optional[r00] last=Optional[r2] 2023-03-08 07:54:51,007 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ce2d248c97134ef7880429885599b03b}] 2023-03-08 07:54:51,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ce2d248c97134ef7880429885599b03b for inclusion in 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:51,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 07:54:51,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:51,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:51,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ce2d248c97134ef7880429885599b03b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__tbgn2hi2o1iivjpl873ch8cv2kk5jtq5iaeaf39de4lqmnf7t00r5ctsb4etu2h8/family1/ce2d248c97134ef7880429885599b03b 2023-03-08 07:54:51,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__tbgn2hi2o1iivjpl873ch8cv2kk5jtq5iaeaf39de4lqmnf7t00r5ctsb4etu2h8/family1/ce2d248c97134ef7880429885599b03b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_ 2023-03-08 07:54:51,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__tbgn2hi2o1iivjpl873ch8cv2kk5jtq5iaeaf39de4lqmnf7t00r5ctsb4etu2h8/family1/ce2d248c97134ef7880429885599b03b into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_ - updating store file list. 2023-03-08 07:54:51,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_ into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:51,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__tbgn2hi2o1iivjpl873ch8cv2kk5jtq5iaeaf39de4lqmnf7t00r5ctsb4etu2h8/family1/ce2d248c97134ef7880429885599b03b into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_) 2023-03-08 07:54:51,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__tbgn2hi2o1iivjpl873ch8cv2kk5jtq5iaeaf39de4lqmnf7t00r5ctsb4etu2h8/family1/ce2d248c97134ef7880429885599b03b 2023-03-08 07:54:51,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 07:54:51,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 07:54:51,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 is initiating minor compaction (all files) 2023-03-08 07:54:51,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@22ee03cb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:51,020 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:51,020 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b5042a4b041b4e7ea9500d4050061cc0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/df99e25d9ba84580a338fa7aab378b40, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp, totalSize=390.1 K 2023-03-08 07:54:51,022 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting b5042a4b041b4e7ea9500d4050061cc0, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839991 2023-03-08 07:54:51,027 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting df99e25d9ba84580a338fa7aab378b40, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239991 2023-03-08 07:54:51,029 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 0235a5be71fe4895ab419a7975003437_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839991 2023-03-08 07:54:51,030 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:51,034 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/769f7f31416c458db35dde7d9b7839ee, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/9df9c0c797f745be80ad313a0a478ed5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/c5eb397720a14bac94b218cb6f830ee2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/f9fc960b45b842fc85ced1a1619d382c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/583ae279f3bc4d708585d76879378bdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/d0708b62cfab4b91af1cab8061857c28, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/71dfeb61cb5c4165939f49defb2b3a18, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/118131eaefaa44419aeb9dd9617d0e53, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b3b5f00378104cfa936cae1a133d1fa6, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_] to archive 2023-03-08 07:54:51,034 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:51,034 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:51,036 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130dde24e25c9f54f6b9da30d46a6851b3c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130dde24e25c9f54f6b9da30d46a6851b3c 2023-03-08 07:54:51,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/769f7f31416c458db35dde7d9b7839ee to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/769f7f31416c458db35dde7d9b7839ee 2023-03-08 07:54:51,037 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201f5b1752137774e90b327e63aa48dfd7f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201f5b1752137774e90b327e63aa48dfd7f 2023-03-08 07:54:51,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/9df9c0c797f745be80ad313a0a478ed5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/9df9c0c797f745be80ad313a0a478ed5 2023-03-08 07:54:51,039 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205df20c73486e34a7dad426692dfeb279b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205df20c73486e34a7dad426692dfeb279b 2023-03-08 07:54:51,040 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-08 07:54:51,040 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 07:54:51,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/c5eb397720a14bac94b218cb6f830ee2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/c5eb397720a14bac94b218cb6f830ee2 2023-03-08 07:54:51,041 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:51,041 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:51,041 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:51,042 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:51,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/f9fc960b45b842fc85ced1a1619d382c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/f9fc960b45b842fc85ced1a1619d382c 2023-03-08 07:54:51,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8590 sec 2023-03-08 07:54:51,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/583ae279f3bc4d708585d76879378bdc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/583ae279f3bc4d708585d76879378bdc 2023-03-08 07:54:51,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/d0708b62cfab4b91af1cab8061857c28 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/d0708b62cfab4b91af1cab8061857c28 2023-03-08 07:54:51,048 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/71dfeb61cb5c4165939f49defb2b3a18 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/71dfeb61cb5c4165939f49defb2b3a18 2023-03-08 07:54:51,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/118131eaefaa44419aeb9dd9617d0e53 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/118131eaefaa44419aeb9dd9617d0e53 2023-03-08 07:54:51,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b3b5f00378104cfa936cae1a133d1fa6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b3b5f00378104cfa936cae1a133d1fa6 2023-03-08 07:54:51,052 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 88f4fcc1aaf1e4b2430d092d92ce36a8#family1#compaction#61 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 76.92 MB/second 2023-03-08 07:54:51,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/de1dfbc6f7f9400aa93177295da8ec1e_SeqId_44_ 2023-03-08 07:54:51,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 07:54:51,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:51,537 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 07:54:51,538 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 07:54:51,538 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 07:54:51,540 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 07:54:51,543 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE}] 2023-03-08 07:54:51,543 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE 2023-03-08 07:54:51,544 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:51,544 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262091544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262091544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262091544"}]},"ts":"1678262091544"} 2023-03-08 07:54:51,545 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:51,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:51,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 88f4fcc1aaf1e4b2430d092d92ce36a8, disabling compactions & flushes 2023-03-08 07:54:51,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,274 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/.tmp/family1/b56a9136a80041b8bdd4654c870044cb as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b56a9136a80041b8bdd4654c870044cb 2023-03-08 07:54:52,279 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 of 88f4fcc1aaf1e4b2430d092d92ce36a8 into b56a9136a80041b8bdd4654c870044cb(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:54:52,279 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:52,279 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,279 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1678262091019; duration=1sec 2023-03-08 07:54:52,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. after waiting 0 ms 2023-03-08 07:54:52,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,279 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:54:52,280 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b5042a4b041b4e7ea9500d4050061cc0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/df99e25d9ba84580a338fa7aab378b40, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_] to archive 2023-03-08 07:54:52,281 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:52,282 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b5042a4b041b4e7ea9500d4050061cc0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b5042a4b041b4e7ea9500d4050061cc0 2023-03-08 07:54:52,284 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/df99e25d9ba84580a338fa7aab378b40 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/df99e25d9ba84580a338fa7aab378b40 2023-03-08 07:54:52,285 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/0235a5be71fe4895ab419a7975003437_SeqId_47_ 2023-03-08 07:54:52,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 07:54:52,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:52,291 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 88f4fcc1aaf1e4b2430d092d92ce36a8 to self. 2023-03-08 07:54:52,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,292 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=CLOSED 2023-03-08 07:54:52,292 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262092292"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262092292"}]},"ts":"1678262092292"} 2023-03-08 07:54:52,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-08 07:54:52,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 749 msec 2023-03-08 07:54:52,295 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=true 2023-03-08 07:54:52,446 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:52,446 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262092446"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262092446"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262092446"}]},"ts":"1678262092446"} 2023-03-08 07:54:52,448 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:52,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 88f4fcc1aaf1e4b2430d092d92ce36a8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:54:52,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:52,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,604 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,605 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:52,605 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:52,605 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88f4fcc1aaf1e4b2430d092d92ce36a8 columnFamilyName family1 2023-03-08 07:54:52,610 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b56a9136a80041b8bdd4654c870044cb 2023-03-08 07:54:52,610 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(310): Store=88f4fcc1aaf1e4b2430d092d92ce36a8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:52,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:52,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 88f4fcc1aaf1e4b2430d092d92ce36a8; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68583806, jitterRate=0.02197834849357605}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:54:52,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:52,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., pid=47, masterSystemTime=1678262092599 2023-03-08 07:54:52,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:52,617 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:52,617 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262092617"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262092617"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262092617"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262092617"}]},"ts":"1678262092617"} 2023-03-08 07:54:52,619 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-08 07:54:52,619 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 170 msec 2023-03-08 07:54:52,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-08 07:54:52,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE in 1.0760 sec 2023-03-08 07:54:52,623 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-08 07:54:52,623 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0810 sec 2023-03-08 07:54:52,625 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.4970 sec 2023-03-08 07:54:52,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-08 07:54:52,627 INFO [Listener at localhost/42999] compactions.TestMobCompactor(278): alter status finished 2023-03-08 07:54:52,628 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:52,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:52,629 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:52,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:52,630 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262692630 2023-03-08 07:54:52,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:54:52,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 07:54:52,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:52,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x567d6105 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:52,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@163eecbc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:52,636 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:52,636 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:52,637 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:52,637 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:52,637 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:53,465 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511281d3f20f61e9944839eb806a23c76cfd9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511281d3f20f61e9944839eb806a23c76cfd9 2023-03-08 07:54:53,466 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:53,478 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:54:53,479 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35834, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:54:53,485 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:54:53,487 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55136, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:54:53,495 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c8a248808b204ef58a9a06407fc94e48 first=Optional[r05] last=Optional[r8] 2023-03-08 07:54:53,495 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c8a248808b204ef58a9a06407fc94e48}] 2023-03-08 07:54:53,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c8a248808b204ef58a9a06407fc94e48 for inclusion in 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:53,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 07:54:53,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:54:53,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:53,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c8a248808b204ef58a9a06407fc94e48 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2kkt7visuu31nppirq12jje1ahu6g67th014k3pqe56svp480ck2o58rd9v50hq0/family1/c8a248808b204ef58a9a06407fc94e48 2023-03-08 07:54:53,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2kkt7visuu31nppirq12jje1ahu6g67th014k3pqe56svp480ck2o58rd9v50hq0/family1/c8a248808b204ef58a9a06407fc94e48 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/1b5e423839ab4a36b880aa094fa27312_SeqId_53_ 2023-03-08 07:54:53,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2kkt7visuu31nppirq12jje1ahu6g67th014k3pqe56svp480ck2o58rd9v50hq0/family1/c8a248808b204ef58a9a06407fc94e48 into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/1b5e423839ab4a36b880aa094fa27312_SeqId_53_ - updating store file list. 2023-03-08 07:54:53,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/1b5e423839ab4a36b880aa094fa27312_SeqId_53_ into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:53,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2kkt7visuu31nppirq12jje1ahu6g67th014k3pqe56svp480ck2o58rd9v50hq0/family1/c8a248808b204ef58a9a06407fc94e48 into 88f4fcc1aaf1e4b2430d092d92ce36a8/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/1b5e423839ab4a36b880aa094fa27312_SeqId_53_) 2023-03-08 07:54:53,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2kkt7visuu31nppirq12jje1ahu6g67th014k3pqe56svp480ck2o58rd9v50hq0/family1/c8a248808b204ef58a9a06407fc94e48 2023-03-08 07:54:53,513 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:53,514 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:54:53,515 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112068dcaaa54c474545b4ab082913e30060 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112068dcaaa54c474545b4ab082913e30060 2023-03-08 07:54:53,516 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284764b15b415f47c8b07cf0beaddbfda0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284764b15b415f47c8b07cf0beaddbfda0 2023-03-08 07:54:53,516 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-08 07:54:53,517 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 07:54:53,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:53,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:53,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:53,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:53,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:53,518 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:53,520 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 890 msec 2023-03-08 07:54:53,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 07:54:53,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:53,974 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 07:54:53,975 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 07:54:53,975 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 07:54:53,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 07:54:53,980 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE}] 2023-03-08 07:54:53,981 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE 2023-03-08 07:54:53,981 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:53,981 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262093981"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262093981"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262093981"}]},"ts":"1678262093981"} 2023-03-08 07:54:53,983 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:54,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 88f4fcc1aaf1e4b2430d092d92ce36a8, disabling compactions & flushes 2023-03-08 07:54:54,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. after waiting 0 ms 2023-03-08 07:54:54,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 07:54:54,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:54,142 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 88f4fcc1aaf1e4b2430d092d92ce36a8 to self. 2023-03-08 07:54:54,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,144 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=CLOSED 2023-03-08 07:54:54,144 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262094143"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262094143"}]},"ts":"1678262094143"} 2023-03-08 07:54:54,146 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-08 07:54:54,146 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 162 msec 2023-03-08 07:54:54,147 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=true 2023-03-08 07:54:54,297 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:54,298 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262094297"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262094297"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262094297"}]},"ts":"1678262094297"} 2023-03-08 07:54:54,299 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:54,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 88f4fcc1aaf1e4b2430d092d92ce36a8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:54:54,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:54,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,455 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,456 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:54,457 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1 2023-03-08 07:54:54,457 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88f4fcc1aaf1e4b2430d092d92ce36a8 columnFamilyName family1 2023-03-08 07:54:54,462 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/1b5e423839ab4a36b880aa094fa27312_SeqId_53_ 2023-03-08 07:54:54,465 DEBUG [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/family1/b56a9136a80041b8bdd4654c870044cb 2023-03-08 07:54:54,465 INFO [StoreOpener-88f4fcc1aaf1e4b2430d092d92ce36a8-1] regionserver.HStore(310): Store=88f4fcc1aaf1e4b2430d092d92ce36a8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:54,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:54:54,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 88f4fcc1aaf1e4b2430d092d92ce36a8; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62030037, jitterRate=-0.07568041980266571}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:54:54,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:54:54,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8., pid=53, masterSystemTime=1678262094451 2023-03-08 07:54:54,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:54:54,473 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=88f4fcc1aaf1e4b2430d092d92ce36a8, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:54,473 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678262094473"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262094473"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262094473"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262094473"}]},"ts":"1678262094473"} 2023-03-08 07:54:54,476 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-08 07:54:54,476 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 88f4fcc1aaf1e4b2430d092d92ce36a8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 175 msec 2023-03-08 07:54:54,477 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-08 07:54:54,477 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=88f4fcc1aaf1e4b2430d092d92ce36a8, REOPEN/MOVE in 496 msec 2023-03-08 07:54:54,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-08 07:54:54,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 501 msec 2023-03-08 07:54:54,481 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 916 msec 2023-03-08 07:54:54,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-08 07:54:54,483 INFO [Listener at localhost/42999] compactions.TestMobCompactor(278): alter status finished 2023-03-08 07:54:54,484 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 07:54:54,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:54,486 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:54,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:54,486 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262694486 2023-03-08 07:54:54,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:54:54,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 07:54:54,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:54,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x71187284 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:54:54,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10a23b17, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 07:54:54,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 07:54:54,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:54:54,494 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 07:54:54,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-03-08 07:54:54,538 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=242 (was 255), OpenFileDescriptor=615 (was 614) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 250), ProcessCount=170 (was 170), AvailableMemoryMB=7582 (was 7588) 2023-03-08 07:54:54,549 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242, OpenFileDescriptor=615, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=170, AvailableMemoryMB=7581 2023-03-08 07:54:54,551 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 07:54:54,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:54,553 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:54:54,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-08 07:54:54,553 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:54:54,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 07:54:54,555 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:54,555 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 empty. 2023-03-08 07:54:54,555 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:54,556 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-08 07:54:54,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 07:54:54,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 07:54:54,963 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 07:54:54,964 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => b63cf1f82efef5926c4509ac90187cb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:54:54,969 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:54,969 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing b63cf1f82efef5926c4509ac90187cb1, disabling compactions & flushes 2023-03-08 07:54:54,970 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:54,970 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:54,970 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. after waiting 0 ms 2023-03-08 07:54:54,970 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:54,970 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:54,970 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:54,972 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:54:54,972 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262094972"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262094972"}]},"ts":"1678262094972"} 2023-03-08 07:54:54,974 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:54:54,974 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:54:54,974 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262094974"}]},"ts":"1678262094974"} 2023-03-08 07:54:54,975 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-08 07:54:54,978 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, ASSIGN}] 2023-03-08 07:54:54,979 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, ASSIGN 2023-03-08 07:54:54,980 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:54:55,131 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,131 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262095131"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262095131"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262095131"}]},"ts":"1678262095131"} 2023-03-08 07:54:55,132 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:54:55,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 07:54:55,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:55,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b63cf1f82efef5926c4509ac90187cb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:54:55,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:54:55,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,289 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,290 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:54:55,290 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:54:55,291 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b63cf1f82efef5926c4509ac90187cb1 columnFamilyName family1 2023-03-08 07:54:55,292 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(310): Store=b63cf1f82efef5926c4509ac90187cb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:54:55,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:54:55,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:54:55,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b63cf1f82efef5926c4509ac90187cb1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61093995, jitterRate=-0.08962853252887726}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:54:55,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:55,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., pid=57, masterSystemTime=1678262095284 2023-03-08 07:54:55,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:55,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:55,301 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,301 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262095301"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262095301"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262095301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262095301"}]},"ts":"1678262095301"} 2023-03-08 07:54:55,304 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-08 07:54:55,304 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 170 msec 2023-03-08 07:54:55,306 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-08 07:54:55,306 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, ASSIGN in 326 msec 2023-03-08 07:54:55,307 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:54:55,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262095307"}]},"ts":"1678262095307"} 2023-03-08 07:54:55,308 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-08 07:54:55,313 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:54:55,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 762 msec 2023-03-08 07:54:55,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 07:54:55,659 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-08 07:54:55,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:55,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,685 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:55,685 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:55,685 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:54:55,685 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:54:55,686 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,686 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,687 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,687 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:55,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:55,688 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,688 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,689 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:54:55,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:55,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:54:55,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:55,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:55,696 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:55,696 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:54:55,697 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:54:55,718 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b3595911be2c434f96c4a5d0cc277753 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130b3595911be2c434f96c4a5d0cc277753 2023-03-08 07:54:55,721 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/d94666d79f874fc88a08dc138e3eb4bc 2023-03-08 07:54:55,726 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/d94666d79f874fc88a08dc138e3eb4bc as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/d94666d79f874fc88a08dc138e3eb4bc 2023-03-08 07:54:55,730 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/d94666d79f874fc88a08dc138e3eb4bc, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-08 07:54:55,730 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 33ms, sequenceid=5, compaction requested=false 2023-03-08 07:54:55,730 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:55,731 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:55,731 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:55,731 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:55,731 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:55,731 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,731 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:54:55,731 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:54:55,733 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,733 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:55,733 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,733 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:55,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:54:55,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@564c8cd9[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:55,734 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:54:55,735 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,736 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,736 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,736 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:55,736 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:54:55,736 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:54:55,736 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:55,738 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,738 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:54:55,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:55,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:55,738 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:55,739 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:54:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:55,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:54:55,741 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:54:55,741 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:55,741 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:55,741 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,741 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:55,742 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,742 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:55,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,742 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:55,743 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:55,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,744 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:55,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,750 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,750 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:55,750 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:55,750 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:55,750 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:55,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:55,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,751 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:55,851 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:55,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:55,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:55,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:55,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,858 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:55,858 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:55,858 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:54:55,858 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:54:55,859 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,859 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,860 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:55,860 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:55,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:55,860 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:55,860 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:55,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:55,861 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:55,862 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:54:55,862 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:54:56,274 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112818a3883b7047489ea9cc17d9b42d2d14 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112818a3883b7047489ea9cc17d9b42d2d14 2023-03-08 07:54:56,680 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/6f5b95d053304c25b844958c8170630b 2023-03-08 07:54:56,685 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/6f5b95d053304c25b844958c8170630b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/6f5b95d053304c25b844958c8170630b 2023-03-08 07:54:56,688 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/6f5b95d053304c25b844958c8170630b, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:54:56,689 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 827ms, sequenceid=9, compaction requested=false 2023-03-08 07:54:56,689 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:56,689 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:56,689 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:56,689 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:56,689 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,689 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:54:56,689 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:54:56,691 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,691 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:56,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:56,691 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,692 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:56,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:56,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:56,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:56,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:54:56,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1487251b[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:56,693 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:54:56,693 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,695 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:54:56,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:54:56,696 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,696 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:56,696 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:56,697 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:54:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:54:56,700 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:54:56,700 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:56,700 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:56,700 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,700 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:56,701 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,701 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:56,701 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:56,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:56,702 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:56,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,703 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:56,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:56,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:56,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,705 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:56,706 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:56,705 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:56,706 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:56,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:56,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:56,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:56,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:56,806 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:56,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:56,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:56,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:56,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,812 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:56,812 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:56,812 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:54:56,812 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:54:56,812 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,812 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,814 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,814 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:56,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:56,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:56,814 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:56,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,814 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:56,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:56,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:54:56,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:56,815 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:54:56,815 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:56,815 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:56,816 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:56,816 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:54:56,816 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:57,229 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201549d6ec25ec04cc9a8bf6e083b536eea to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201549d6ec25ec04cc9a8bf6e083b536eea 2023-03-08 07:54:57,235 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/f1a968e0e0c94b3483fe482566f1681b 2023-03-08 07:54:57,238 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/f1a968e0e0c94b3483fe482566f1681b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/f1a968e0e0c94b3483fe482566f1681b 2023-03-08 07:54:57,242 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/f1a968e0e0c94b3483fe482566f1681b, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 07:54:57,244 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 428ms, sequenceid=13, compaction requested=true 2023-03-08 07:54:57,244 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:57,244 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:57,244 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:54:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:54:57,246 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,246 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:57,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:57,246 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,246 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:57,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:57,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:57,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:57,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:54:57,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f665488[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:57,248 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:54:57,248 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,249 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,249 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:54:57,249 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:54:57,249 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:57,249 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,249 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:57,250 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:54:57,250 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,250 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:57,251 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:54:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:57,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:54:57,253 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:54:57,253 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:57,253 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:57,253 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,253 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:57,255 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,255 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:57,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,255 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:57,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:57,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:57,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:57,256 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:57,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,257 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:57,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:57,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:57,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:57,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:57,259 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:57,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:57,260 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:57,260 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:57,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:57,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:57,360 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:57,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:57,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:57,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:57,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,366 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:57,366 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:57,366 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:54:57,366 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:54:57,367 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,367 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,368 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:57,368 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:57,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:57,368 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:57,368 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:57,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:57,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:54:57,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:57,370 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:54:57,370 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:57,370 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:54:57,370 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:57,370 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:57,370 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:57,370 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:54:57,370 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:57,786 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f7e9f5c9dfca4be4a4d8f99575a4062e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205f7e9f5c9dfca4be4a4d8f99575a4062e 2023-03-08 07:54:57,830 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:54:58,192 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/3dbbc365eb314e6891114a83d2bbd8a1 2023-03-08 07:54:58,197 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/3dbbc365eb314e6891114a83d2bbd8a1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/3dbbc365eb314e6891114a83d2bbd8a1 2023-03-08 07:54:58,200 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/3dbbc365eb314e6891114a83d2bbd8a1, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 07:54:58,201 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 831ms, sequenceid=17, compaction requested=true 2023-03-08 07:54:58,201 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:58,201 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:58,201 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:58,201 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:58,201 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:54:58,203 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:58,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:54:58,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ac7c116[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:58,205 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:54:58,205 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,209 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:54:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:54:58,209 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,209 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:58,211 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,211 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:54:58,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:58,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:58,211 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:58,211 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:54:58,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:58,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:58,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:58,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:58,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:54:58,213 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:54:58,213 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:58,213 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:58,213 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,213 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:58,214 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,214 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:58,214 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,215 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:58,215 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,218 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,218 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:58,218 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:58,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:58,218 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:58,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:58,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:58,218 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:58,218 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:54:58,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:58,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:58,219 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:58,219 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:58,219 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,219 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,219 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,219 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,319 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:58,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:58,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:58,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:58,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,325 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:58,325 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:58,325 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:54:58,325 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:54:58,326 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,326 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,327 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,327 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:58,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:58,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:58,327 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:58,327 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:58,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,328 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:58,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:54:58,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:58,328 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:54:58,328 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:58,328 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:54:58,329 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:58,329 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:58,329 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:58,329 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:54:58,329 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:58,740 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228b0701d9ef120439aa10eb6064886fb1a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228b0701d9ef120439aa10eb6064886fb1a 2023-03-08 07:54:59,146 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/1d5df01809344605941c2a6cd69c6663 2023-03-08 07:54:59,150 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/1d5df01809344605941c2a6cd69c6663 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/1d5df01809344605941c2a6cd69c6663 2023-03-08 07:54:59,154 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/1d5df01809344605941c2a6cd69c6663, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 07:54:59,155 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 826ms, sequenceid=21, compaction requested=true 2023-03-08 07:54:59,155 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:54:59,155 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:59,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:54:59,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:54:59,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:54:59,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:54:59,156 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,157 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:59,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:59,157 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,157 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:54:59,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:59,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:59,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@385faed3[Count = 0] remaining members to acquire global barrier 2023-03-08 07:54:59,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:54:59,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,159 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:54:59,159 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:54:59,159 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:54:59,159 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,159 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:54:59,161 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:54:59,161 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,161 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:59,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:59,161 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:54:59,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:54:59,164 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:54:59,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:54:59,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:54:59,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,164 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:54:59,165 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,165 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:54:59,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:54:59,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,165 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:59,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:54:59,166 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:59,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:59,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:54:59,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,167 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:54:59,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:59,169 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:54:59,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:54:59,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:59,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:54:59,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:54:59,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:54:59,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:54:59,169 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:54:59,269 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:54:59,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:54:59,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:54:59,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:54:59,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,276 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:54:59,276 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:59,276 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:54:59,276 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:54:59,276 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,278 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,278 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:54:59,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:54:59,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:54:59,278 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:54:59,278 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:54:59,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:54:59,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:54:59,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:54:59,279 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:54:59,279 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:54:59,279 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:54:59,279 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:54:59,279 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:54:59,279 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:54:59,279 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:54:59,279 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:54:59,691 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312b8768213ba3452ab2b0dcb33bbebfd5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512312b8768213ba3452ab2b0dcb33bbebfd5 2023-03-08 07:55:00,096 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/ddeebcc9363c4468ae586a033a54f40c 2023-03-08 07:55:00,101 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/ddeebcc9363c4468ae586a033a54f40c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ddeebcc9363c4468ae586a033a54f40c 2023-03-08 07:55:00,106 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ddeebcc9363c4468ae586a033a54f40c, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 07:55:00,107 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 828ms, sequenceid=25, compaction requested=true 2023-03-08 07:55:00,107 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:00,107 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:55:00,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,110 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:00,110 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,110 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:00,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:00,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:00,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:55:00,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15ddeb3e[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:00,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:55:00,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,113 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:55:00,113 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,113 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:00,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:55:00,113 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:00,114 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:55:00,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,114 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:55:00,116 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:55:00,116 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:00,116 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:00,117 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,117 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:00,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:00,119 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:00,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,120 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:00,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,125 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,125 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:00,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:00,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:00,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:55:00,126 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:00,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:00,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:00,126 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:00,226 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:00,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:00,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:00,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:00,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,232 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:00,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:00,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:55:00,232 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:55:00,233 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,233 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,237 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:00,237 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:00,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:00,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:55:00,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:00,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:55:00,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:00,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:55:00,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:55:00,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:00,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:00,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:55:00,239 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:55:00,651 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c08cf407bdbb44ed95083ccd00267b70 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101c08cf407bdbb44ed95083ccd00267b70 2023-03-08 07:55:01,056 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/abfdd44a251f4c73bbf8517529da4242 2023-03-08 07:55:01,062 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/abfdd44a251f4c73bbf8517529da4242 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/abfdd44a251f4c73bbf8517529da4242 2023-03-08 07:55:01,067 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/abfdd44a251f4c73bbf8517529da4242, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 07:55:01,068 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 829ms, sequenceid=29, compaction requested=true 2023-03-08 07:55:01,068 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:01,068 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:55:01,070 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:01,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,070 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68e22f19[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:01,071 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:55:01,072 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,073 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:55:01,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:55:01,073 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,073 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:01,073 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:01,074 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,074 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:55:01,074 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,074 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:55:01,076 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:55:01,076 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:01,076 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:01,077 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,077 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:01,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,079 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:01,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:01,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:01,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:01,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:01,080 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:01,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:01,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,081 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:01,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:01,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:01,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:01,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:01,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:55:01,083 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:01,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,083 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,183 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:01,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:01,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:01,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:01,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,190 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:01,190 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:01,190 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:55:01,190 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:55:01,190 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,190 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,191 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,191 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:01,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:01,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:01,191 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:01,192 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:01,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:01,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:55:01,192 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:01,193 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:55:01,193 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:01,193 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:55:01,193 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:55:01,193 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:01,193 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:01,193 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:55:01,193 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:55:01,606 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ccd55c0768da4041b89b7cf8498cd494 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ccd55c0768da4041b89b7cf8498cd494 2023-03-08 07:55:02,012 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/96007f05b0b244a099750d6dcb1537ce 2023-03-08 07:55:02,016 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/96007f05b0b244a099750d6dcb1537ce as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/96007f05b0b244a099750d6dcb1537ce 2023-03-08 07:55:02,020 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/96007f05b0b244a099750d6dcb1537ce, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 07:55:02,021 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 828ms, sequenceid=33, compaction requested=true 2023-03-08 07:55:02,021 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:02,021 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:55:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:55:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:55:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:55:02,022 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,022 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:02,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:55:02,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c199c62[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:02,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:55:02,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,026 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:55:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:55:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:02,026 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,026 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:02,027 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:55:02,027 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,027 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,027 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:55:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:02,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:55:02,029 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:55:02,029 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:02,029 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:02,029 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,030 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:02,031 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,031 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:02,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,031 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:02,031 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,032 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:02,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:02,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:02,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:02,034 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,035 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,035 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,034 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:02,035 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:02,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:02,035 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:02,135 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:02,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:02,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:02,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:02,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,142 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:02,142 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:02,143 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:55:02,143 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:55:02,143 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,143 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,145 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,145 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:02,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:02,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:02,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,146 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:02,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,146 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:55:02,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:55:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:55:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:55:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:02,147 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:02,147 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:55:02,147 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:55:02,560 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078f9075c8aef14b03b8b9b0dff35b155f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509078f9075c8aef14b03b8b9b0dff35b155f 2023-03-08 07:55:02,965 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/b0a108c8b1d24020af90526ffe576f9d 2023-03-08 07:55:02,969 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/b0a108c8b1d24020af90526ffe576f9d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/b0a108c8b1d24020af90526ffe576f9d 2023-03-08 07:55:02,973 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/b0a108c8b1d24020af90526ffe576f9d, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 07:55:02,976 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 829ms, sequenceid=37, compaction requested=true 2023-03-08 07:55:02,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:02,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:02,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:55:02,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:55:02,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:55:02,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:55:02,978 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:02,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:02,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:55:02,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19758804[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:02,979 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:55:02,979 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:55:02,981 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,981 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:55:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:02,982 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:02,982 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:55:02,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:02,982 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:02,982 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:55:02,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:02,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:55:02,984 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:55:02,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:02,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:02,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,984 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:02,986 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,986 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,986 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:02,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:02,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:02,987 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:02,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:02,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:02,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:02,988 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:02,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:02,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:02,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:02,990 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:02,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:02,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:55:02,990 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:02,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:02,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:03,090 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:03,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:03,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:03,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:03,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,097 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:03,097 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:03,097 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 07:55:03,097 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 07:55:03,098 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,098 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,099 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,099 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:03,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:03,099 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,099 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 07:55:03,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,104 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 07:55:03,104 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 07:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:03,105 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:03,105 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. started... 2023-03-08 07:55:03,105 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b63cf1f82efef5926c4509ac90187cb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:55:03,525 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120cbbf6a14967c4a44afa9d4007ef28175 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120cbbf6a14967c4a44afa9d4007ef28175 2023-03-08 07:55:03,930 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/0abcc3e9d1f74104a523877929d4481d 2023-03-08 07:55:03,935 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/0abcc3e9d1f74104a523877929d4481d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/0abcc3e9d1f74104a523877929d4481d 2023-03-08 07:55:03,939 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/0abcc3e9d1f74104a523877929d4481d, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 07:55:03,939 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b63cf1f82efef5926c4509ac90187cb1 in 834ms, sequenceid=41, compaction requested=true 2023-03-08 07:55:03,940 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:03,940 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 07:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 07:55:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,941 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:03,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 07:55:03,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@638b140f[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:03,943 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 07:55:03,943 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,944 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 07:55:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 07:55:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:03,945 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,945 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:03,946 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,946 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 07:55:03,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:03,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:03,946 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:03,947 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 07:55:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:03,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 07:55:03,949 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 07:55:03,949 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:03,949 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:03,949 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,949 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:03,950 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,950 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:03,950 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:03,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:03,951 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,952 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:03,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,954 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:03,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 07:55:03,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:03,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 07:55:03,955 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:03,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:03,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:03,955 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:03,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:03,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:04,055 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:04,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:04,057 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:04,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:04,060 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:04,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:04,060 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262704060 2023-03-08 07:55:04,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:55:04,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 07:55:04,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:04,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x2110ec23 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:04,068 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@72ebc3af, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:04,068 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,915 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e961a5d16e874473b13d8844d9208e11 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103e961a5d16e874473b13d8844d9208e11 2023-03-08 07:55:04,916 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:04,935 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:04,936 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52258, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:04,945 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:55:04,946 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59944, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:55:04,955 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4ada61ca7f9f4350ac069e721e052aa5 first=Optional[r3] last=Optional[r6] 2023-03-08 07:55:04,956 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4ada61ca7f9f4350ac069e721e052aa5}] 2023-03-08 07:55:04,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4ada61ca7f9f4350ac069e721e052aa5 for inclusion in b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:04,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 07:55:04,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:55:04,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:04,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4ada61ca7f9f4350ac069e721e052aa5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emiisi8j45alr07qc6mbc72lmhhbu0e4i6fqn1nvvolj09irj8fcvuniqm9d4ttr/family1/4ada61ca7f9f4350ac069e721e052aa5 2023-03-08 07:55:04,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emiisi8j45alr07qc6mbc72lmhhbu0e4i6fqn1nvvolj09irj8fcvuniqm9d4ttr/family1/4ada61ca7f9f4350ac069e721e052aa5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_ 2023-03-08 07:55:04,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emiisi8j45alr07qc6mbc72lmhhbu0e4i6fqn1nvvolj09irj8fcvuniqm9d4ttr/family1/4ada61ca7f9f4350ac069e721e052aa5 into b63cf1f82efef5926c4509ac90187cb1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_ - updating store file list. 2023-03-08 07:55:04,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_ into b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:04,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emiisi8j45alr07qc6mbc72lmhhbu0e4i6fqn1nvvolj09irj8fcvuniqm9d4ttr/family1/4ada61ca7f9f4350ac069e721e052aa5 into b63cf1f82efef5926c4509ac90187cb1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_) 2023-03-08 07:55:04,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emiisi8j45alr07qc6mbc72lmhhbu0e4i6fqn1nvvolj09irj8fcvuniqm9d4ttr/family1/4ada61ca7f9f4350ac069e721e052aa5 2023-03-08 07:55:04,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:55:04,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:55:04,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): b63cf1f82efef5926c4509ac90187cb1/family1 is initiating minor compaction 2023-03-08 07:55:04,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4e848ab5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:04,976 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b63cf1f82efef5926c4509ac90187cb1/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:04,976 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/6f5b95d053304c25b844958c8170630b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/f1a968e0e0c94b3483fe482566f1681b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/3dbbc365eb314e6891114a83d2bbd8a1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/1d5df01809344605941c2a6cd69c6663, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ddeebcc9363c4468ae586a033a54f40c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/abfdd44a251f4c73bbf8517529da4242, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/96007f05b0b244a099750d6dcb1537ce, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/b0a108c8b1d24020af90526ffe576f9d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/0abcc3e9d1f74104a523877929d4481d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp, totalSize=50.3 K 2023-03-08 07:55:04,976 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 6f5b95d053304c25b844958c8170630b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:55:04,976 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting f1a968e0e0c94b3483fe482566f1681b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:55:04,976 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 3dbbc365eb314e6891114a83d2bbd8a1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:55:04,976 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 1d5df01809344605941c2a6cd69c6663, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:55:04,976 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ddeebcc9363c4468ae586a033a54f40c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:55:04,976 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting abfdd44a251f4c73bbf8517529da4242, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:55:04,977 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 96007f05b0b244a099750d6dcb1537ce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:55:04,977 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting b0a108c8b1d24020af90526ffe576f9d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:55:04,977 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 0abcc3e9d1f74104a523877929d4481d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:55:04,977 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ae76138c27764c8187a43a81e2426f9c_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:55:04,978 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:04,979 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:04,982 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228b0701d9ef120439aa10eb6064886fb1a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228b0701d9ef120439aa10eb6064886fb1a 2023-03-08 07:55:04,983 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512312b8768213ba3452ab2b0dcb33bbebfd5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512312b8768213ba3452ab2b0dcb33bbebfd5 2023-03-08 07:55:04,984 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101c08cf407bdbb44ed95083ccd00267b70 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101c08cf407bdbb44ed95083ccd00267b70 2023-03-08 07:55:04,985 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ccd55c0768da4041b89b7cf8498cd494 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ccd55c0768da4041b89b7cf8498cd494 2023-03-08 07:55:04,985 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-08 07:55:04,985 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:04,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:05,016 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): b63cf1f82efef5926c4509ac90187cb1#family1#compaction#72 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 76.92 MB/second 2023-03-08 07:55:05,446 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/275f996be8634971b0bde54a11383730 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/275f996be8634971b0bde54a11383730 2023-03-08 07:55:05,450 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b63cf1f82efef5926c4509ac90187cb1/family1 of b63cf1f82efef5926c4509ac90187cb1 into 275f996be8634971b0bde54a11383730(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:55:05,451 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:05,451 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678262104975; duration=0sec 2023-03-08 07:55:05,451 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:05,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/6f5b95d053304c25b844958c8170630b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/f1a968e0e0c94b3483fe482566f1681b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/3dbbc365eb314e6891114a83d2bbd8a1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/1d5df01809344605941c2a6cd69c6663, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ddeebcc9363c4468ae586a033a54f40c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/abfdd44a251f4c73bbf8517529da4242, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/96007f05b0b244a099750d6dcb1537ce, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/b0a108c8b1d24020af90526ffe576f9d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/0abcc3e9d1f74104a523877929d4481d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_] to archive 2023-03-08 07:55:05,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:05,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/6f5b95d053304c25b844958c8170630b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/6f5b95d053304c25b844958c8170630b 2023-03-08 07:55:05,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/f1a968e0e0c94b3483fe482566f1681b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/f1a968e0e0c94b3483fe482566f1681b 2023-03-08 07:55:05,534 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/3dbbc365eb314e6891114a83d2bbd8a1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/3dbbc365eb314e6891114a83d2bbd8a1 2023-03-08 07:55:05,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/1d5df01809344605941c2a6cd69c6663 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/1d5df01809344605941c2a6cd69c6663 2023-03-08 07:55:05,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ddeebcc9363c4468ae586a033a54f40c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ddeebcc9363c4468ae586a033a54f40c 2023-03-08 07:55:05,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/abfdd44a251f4c73bbf8517529da4242 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/abfdd44a251f4c73bbf8517529da4242 2023-03-08 07:55:05,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/96007f05b0b244a099750d6dcb1537ce to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/96007f05b0b244a099750d6dcb1537ce 2023-03-08 07:55:05,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/b0a108c8b1d24020af90526ffe576f9d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/b0a108c8b1d24020af90526ffe576f9d 2023-03-08 07:55:05,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/0abcc3e9d1f74104a523877929d4481d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/0abcc3e9d1f74104a523877929d4481d 2023-03-08 07:55:05,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/ae76138c27764c8187a43a81e2426f9c_SeqId_44_ 2023-03-08 07:55:05,859 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120572206d133bb64f82b55f8be319f90344 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120572206d133bb64f82b55f8be319f90344 2023-03-08 07:55:05,860 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:05,885 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b4f623909974ac59a16981858b175d3 first=Optional[r00] last=Optional[r2] 2023-03-08 07:55:05,886 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b4f623909974ac59a16981858b175d3}] 2023-03-08 07:55:05,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b4f623909974ac59a16981858b175d3 for inclusion in b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:05,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 07:55:05,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:55:05,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:05,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b4f623909974ac59a16981858b175d3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b73hk240upke9icgh3j139rvsc6uqmlb3s6pu3lqksavgon6hpn706lf23v7hk4a/family1/7b4f623909974ac59a16981858b175d3 2023-03-08 07:55:05,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b73hk240upke9icgh3j139rvsc6uqmlb3s6pu3lqksavgon6hpn706lf23v7hk4a/family1/7b4f623909974ac59a16981858b175d3 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_ 2023-03-08 07:55:05,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b73hk240upke9icgh3j139rvsc6uqmlb3s6pu3lqksavgon6hpn706lf23v7hk4a/family1/7b4f623909974ac59a16981858b175d3 into b63cf1f82efef5926c4509ac90187cb1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_ - updating store file list. 2023-03-08 07:55:05,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_ into b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:05,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b73hk240upke9icgh3j139rvsc6uqmlb3s6pu3lqksavgon6hpn706lf23v7hk4a/family1/7b4f623909974ac59a16981858b175d3 into b63cf1f82efef5926c4509ac90187cb1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_) 2023-03-08 07:55:05,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b73hk240upke9icgh3j139rvsc6uqmlb3s6pu3lqksavgon6hpn706lf23v7hk4a/family1/7b4f623909974ac59a16981858b175d3 2023-03-08 07:55:05,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 07:55:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 07:55:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): b63cf1f82efef5926c4509ac90187cb1/family1 is initiating minor compaction (all files) 2023-03-08 07:55:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39759dba; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:05,905 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b63cf1f82efef5926c4509ac90187cb1/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:05,905 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/d94666d79f874fc88a08dc138e3eb4bc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/275f996be8634971b0bde54a11383730, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp, totalSize=388.1 K 2023-03-08 07:55:05,927 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting d94666d79f874fc88a08dc138e3eb4bc, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839991 2023-03-08 07:55:05,927 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71187284 to 127.0.0.1:52387 2023-03-08 07:55:05,927 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:05,927 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 275f996be8634971b0bde54a11383730, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239991 2023-03-08 07:55:05,928 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:05,928 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x567d6105 to 127.0.0.1:52387 2023-03-08 07:55:05,928 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:05,928 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:05,928 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21a6eaa4 to 127.0.0.1:52387 2023-03-08 07:55:05,928 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:05,928 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:05,928 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68bc28e2 to 127.0.0.1:52387 2023-03-08 07:55:05,928 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 702d1547fc5944a78edc99aef7c06f24_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839991 2023-03-08 07:55:05,928 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:05,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:05,930 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:05,932 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130b3595911be2c434f96c4a5d0cc277753 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130b3595911be2c434f96c4a5d0cc277753 2023-03-08 07:55:05,934 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201549d6ec25ec04cc9a8bf6e083b536eea to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201549d6ec25ec04cc9a8bf6e083b536eea 2023-03-08 07:55:05,935 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205f7e9f5c9dfca4be4a4d8f99575a4062e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205f7e9f5c9dfca4be4a4d8f99575a4062e 2023-03-08 07:55:05,935 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-08 07:55:05,935 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 07:55:05,935 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:05,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:05,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:05,937 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:05,939 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8790 sec 2023-03-08 07:55:05,951 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): b63cf1f82efef5926c4509ac90187cb1#family1#compaction#73 average throughput is 19.96 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 76.92 MB/second 2023-03-08 07:55:06,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 07:55:06,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:06,508 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 07:55:06,509 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 07:55:06,509 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 07:55:06,510 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 07:55:06,513 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE}] 2023-03-08 07:55:06,514 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE 2023-03-08 07:55:06,514 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:06,514 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262106514"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262106514"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262106514"}]},"ts":"1678262106514"} 2023-03-08 07:55:06,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:06,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:06,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b63cf1f82efef5926c4509ac90187cb1, disabling compactions & flushes 2023-03-08 07:55:06,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:06,773 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/.tmp/family1/571dba6fc84a44fd9576be7205f88f22 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/571dba6fc84a44fd9576be7205f88f22 2023-03-08 07:55:06,777 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b63cf1f82efef5926c4509ac90187cb1/family1 of b63cf1f82efef5926c4509ac90187cb1 into 571dba6fc84a44fd9576be7205f88f22(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:55:06,777 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:06,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:06,777 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1678262105905; duration=0sec 2023-03-08 07:55:06,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:06,777 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:06,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. after waiting 0 ms 2023-03-08 07:55:06,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:06,780 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/d94666d79f874fc88a08dc138e3eb4bc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/275f996be8634971b0bde54a11383730, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_] to archive 2023-03-08 07:55:06,781 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:06,782 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/d94666d79f874fc88a08dc138e3eb4bc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/d94666d79f874fc88a08dc138e3eb4bc 2023-03-08 07:55:06,783 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/275f996be8634971b0bde54a11383730 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/275f996be8634971b0bde54a11383730 2023-03-08 07:55:06,784 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/702d1547fc5944a78edc99aef7c06f24_SeqId_47_ 2023-03-08 07:55:06,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 07:55:06,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:06,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:06,789 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b63cf1f82efef5926c4509ac90187cb1 to self. 2023-03-08 07:55:06,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:06,791 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=CLOSED 2023-03-08 07:55:06,791 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262106791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262106791"}]},"ts":"1678262106791"} 2023-03-08 07:55:06,793 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-08 07:55:06,793 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 276 msec 2023-03-08 07:55:06,793 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=true 2023-03-08 07:55:06,944 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:06,944 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262106944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262106944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262106944"}]},"ts":"1678262106944"} 2023-03-08 07:55:06,946 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:07,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b63cf1f82efef5926c4509ac90187cb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:55:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:07,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,102 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,103 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:07,103 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:07,103 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b63cf1f82efef5926c4509ac90187cb1 columnFamilyName family1 2023-03-08 07:55:07,108 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/571dba6fc84a44fd9576be7205f88f22 2023-03-08 07:55:07,108 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(310): Store=b63cf1f82efef5926c4509ac90187cb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:07,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:07,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b63cf1f82efef5926c4509ac90187cb1; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69107094, jitterRate=0.029775947332382202}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:55:07,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:07,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., pid=63, masterSystemTime=1678262107097 2023-03-08 07:55:07,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:07,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:07,115 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:07,115 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262107115"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262107115"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262107115"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262107115"}]},"ts":"1678262107115"} 2023-03-08 07:55:07,117 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-08 07:55:07,117 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 170 msec 2023-03-08 07:55:07,118 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-08 07:55:07,119 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE in 604 msec 2023-03-08 07:55:07,120 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-08 07:55:07,120 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 609 msec 2023-03-08 07:55:07,122 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0240 sec 2023-03-08 07:55:07,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-08 07:55:07,124 INFO [Listener at localhost/42999] compactions.TestMobCompactor(278): alter status finished 2023-03-08 07:55:07,124 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:07,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:07,126 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:07,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:07,127 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262707127 2023-03-08 07:55:07,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:55:07,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 07:55:07,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:07,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x10a2a106 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:07,136 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a60a3f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:07,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:07,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:07,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:07,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:07,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:07,960 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e350030718bb48cd9e00d2bc2b1c3566 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128e350030718bb48cd9e00d2bc2b1c3566 2023-03-08 07:55:07,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:07,973 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:07,974 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52266, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:07,979 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:55:07,981 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59956, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:55:07,989 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/575388a4bb984f71888eca95d40cc9d0 first=Optional[r05] last=Optional[r8] 2023-03-08 07:55:07,990 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/575388a4bb984f71888eca95d40cc9d0}] 2023-03-08 07:55:07,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/575388a4bb984f71888eca95d40cc9d0 for inclusion in b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:07,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 07:55:07,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:55:07,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:07,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/575388a4bb984f71888eca95d40cc9d0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q64kolqdadj1mckufpd2p1fksk5im6pf1osbttfgm3c6v5q7ohvb1nh2h9tbqeb4/family1/575388a4bb984f71888eca95d40cc9d0 2023-03-08 07:55:07,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q64kolqdadj1mckufpd2p1fksk5im6pf1osbttfgm3c6v5q7ohvb1nh2h9tbqeb4/family1/575388a4bb984f71888eca95d40cc9d0 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/446552e496734705b7403269d8573f1f_SeqId_53_ 2023-03-08 07:55:07,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q64kolqdadj1mckufpd2p1fksk5im6pf1osbttfgm3c6v5q7ohvb1nh2h9tbqeb4/family1/575388a4bb984f71888eca95d40cc9d0 into b63cf1f82efef5926c4509ac90187cb1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/446552e496734705b7403269d8573f1f_SeqId_53_ - updating store file list. 2023-03-08 07:55:08,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/446552e496734705b7403269d8573f1f_SeqId_53_ into b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:08,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q64kolqdadj1mckufpd2p1fksk5im6pf1osbttfgm3c6v5q7ohvb1nh2h9tbqeb4/family1/575388a4bb984f71888eca95d40cc9d0 into b63cf1f82efef5926c4509ac90187cb1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/446552e496734705b7403269d8573f1f_SeqId_53_) 2023-03-08 07:55:08,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q64kolqdadj1mckufpd2p1fksk5im6pf1osbttfgm3c6v5q7ohvb1nh2h9tbqeb4/family1/575388a4bb984f71888eca95d40cc9d0 2023-03-08 07:55:08,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:08,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:08,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120cbbf6a14967c4a44afa9d4007ef28175 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120cbbf6a14967c4a44afa9d4007ef28175 2023-03-08 07:55:08,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112818a3883b7047489ea9cc17d9b42d2d14 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112818a3883b7047489ea9cc17d9b42d2d14 2023-03-08 07:55:08,008 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-08 07:55:08,008 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 07:55:08,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:08,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:08,009 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:08,011 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 884 msec 2023-03-08 07:55:08,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-08 07:55:08,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:08,462 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 07:55:08,463 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 07:55:08,463 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 07:55:08,465 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 07:55:08,467 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE}] 2023-03-08 07:55:08,468 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE 2023-03-08 07:55:08,468 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:08,468 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262108468"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262108468"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262108468"}]},"ts":"1678262108468"} 2023-03-08 07:55:08,469 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:08,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b63cf1f82efef5926c4509ac90187cb1, disabling compactions & flushes 2023-03-08 07:55:08,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. after waiting 0 ms 2023-03-08 07:55:08,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 07:55:08,628 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:08,628 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b63cf1f82efef5926c4509ac90187cb1 to self. 2023-03-08 07:55:08,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,630 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=CLOSED 2023-03-08 07:55:08,630 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262108630"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262108630"}]},"ts":"1678262108630"} 2023-03-08 07:55:08,632 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-08 07:55:08,632 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 162 msec 2023-03-08 07:55:08,633 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=true 2023-03-08 07:55:08,783 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:08,783 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262108783"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262108783"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262108783"}]},"ts":"1678262108783"} 2023-03-08 07:55:08,785 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:08,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b63cf1f82efef5926c4509ac90187cb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:55:08,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:08,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,940 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,941 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:08,941 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1 2023-03-08 07:55:08,941 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b63cf1f82efef5926c4509ac90187cb1 columnFamilyName family1 2023-03-08 07:55:08,945 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/446552e496734705b7403269d8573f1f_SeqId_53_ 2023-03-08 07:55:08,948 DEBUG [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/family1/571dba6fc84a44fd9576be7205f88f22 2023-03-08 07:55:08,948 INFO [StoreOpener-b63cf1f82efef5926c4509ac90187cb1-1] regionserver.HStore(310): Store=b63cf1f82efef5926c4509ac90187cb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:08,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:55:08,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b63cf1f82efef5926c4509ac90187cb1; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66141227, jitterRate=-0.014418914914131165}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:55:08,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:55:08,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1., pid=69, masterSystemTime=1678262108936 2023-03-08 07:55:08,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:55:08,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b63cf1f82efef5926c4509ac90187cb1, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:08,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678262108954"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262108954"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262108954"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262108954"}]},"ts":"1678262108954"} 2023-03-08 07:55:08,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-08 07:55:08,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure b63cf1f82efef5926c4509ac90187cb1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 170 msec 2023-03-08 07:55:08,958 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-08 07:55:08,958 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b63cf1f82efef5926c4509ac90187cb1, REOPEN/MOVE in 490 msec 2023-03-08 07:55:08,959 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-08 07:55:08,959 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 494 msec 2023-03-08 07:55:08,961 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 906 msec 2023-03-08 07:55:08,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-08 07:55:08,962 INFO [Listener at localhost/42999] compactions.TestMobCompactor(278): alter status finished 2023-03-08 07:55:08,963 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 07:55:08,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:08,965 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:08,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:08,965 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262708965 2023-03-08 07:55:08,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:55:08,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 07:55:08,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:08,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x0526bd06 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:08,977 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18acc617, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 07:55:08,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 07:55:08,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:08,978 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 07:55:08,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 16 msec 2023-03-08 07:55:09,024 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240 (was 242), OpenFileDescriptor=620 (was 615) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=217 (was 227), ProcessCount=170 (was 170), AvailableMemoryMB=7555 (was 7581) 2023-03-08 07:55:09,036 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=217, ProcessCount=170, AvailableMemoryMB=7555 2023-03-08 07:55:09,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 07:55:09,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:09,042 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:55:09,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-08 07:55:09,042 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:55:09,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 07:55:09,043 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:09,044 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b empty. 2023-03-08 07:55:09,044 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:09,044 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 07:55:09,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 07:55:09,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 07:55:09,452 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 07:55:09,453 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 240ad2a18a711975ba6b76a83e94017b, NAME => 'testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:09,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 07:55:09,858 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:09,859 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 240ad2a18a711975ba6b76a83e94017b, disabling compactions & flushes 2023-03-08 07:55:09,859 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:09,859 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:09,859 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. after waiting 0 ms 2023-03-08 07:55:09,859 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:09,859 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:09,859 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:09,861 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:55:09,862 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678262109862"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262109862"}]},"ts":"1678262109862"} 2023-03-08 07:55:09,863 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:55:09,863 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:55:09,863 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262109863"}]},"ts":"1678262109863"} 2023-03-08 07:55:09,864 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-08 07:55:09,867 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, ASSIGN}] 2023-03-08 07:55:09,868 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, ASSIGN 2023-03-08 07:55:09,869 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:10,020 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=240ad2a18a711975ba6b76a83e94017b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:10,020 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678262110020"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262110020"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262110020"}]},"ts":"1678262110020"} 2023-03-08 07:55:10,022 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 240ad2a18a711975ba6b76a83e94017b, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:10,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 07:55:10,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:10,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 240ad2a18a711975ba6b76a83e94017b, NAME => 'testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:55:10,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-08 07:55:10,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-08 07:55:10,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:10,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,179 INFO [StoreOpener-240ad2a18a711975ba6b76a83e94017b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,180 DEBUG [StoreOpener-240ad2a18a711975ba6b76a83e94017b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1 2023-03-08 07:55:10,180 DEBUG [StoreOpener-240ad2a18a711975ba6b76a83e94017b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1 2023-03-08 07:55:10,180 INFO [StoreOpener-240ad2a18a711975ba6b76a83e94017b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 240ad2a18a711975ba6b76a83e94017b columnFamilyName f1 2023-03-08 07:55:10,182 INFO [StoreOpener-240ad2a18a711975ba6b76a83e94017b-1] regionserver.HStore(310): Store=240ad2a18a711975ba6b76a83e94017b/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:10,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:10,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:10,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 240ad2a18a711975ba6b76a83e94017b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62602315, jitterRate=-0.06715281307697296}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:55:10,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:10,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b., pid=73, masterSystemTime=1678262110173 2023-03-08 07:55:10,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:10,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:10,189 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=240ad2a18a711975ba6b76a83e94017b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:10,189 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678262110189"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262110189"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262110189"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262110189"}]},"ts":"1678262110189"} 2023-03-08 07:55:10,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-08 07:55:10,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 240ad2a18a711975ba6b76a83e94017b, server=jenkins-hbase4.apache.org,33669,1678262016084 in 168 msec 2023-03-08 07:55:10,192 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-08 07:55:10,193 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, ASSIGN in 324 msec 2023-03-08 07:55:10,193 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:55:10,193 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262110193"}]},"ts":"1678262110193"} 2023-03-08 07:55:10,194 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-08 07:55:10,196 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:55:10,197 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1560 sec 2023-03-08 07:55:11,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 07:55:11,148 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-08 07:55:11,148 DEBUG [Listener at localhost/42999] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-08 07:55:11,150 INFO [Listener at localhost/42999] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 07:55:11,152 INFO [Listener at localhost/42999] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-08 07:55:11,153 INFO [Listener at localhost/42999] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 07:55:11,153 INFO [Listener at localhost/42999] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-08 07:55:11,157 INFO [Listener at localhost/42999] regionserver.HRegion(2745): Flushing 240ad2a18a711975ba6b76a83e94017b 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 07:55:11,572 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303089cf195061c894659947ed5f8c9a93c60 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303089cf195061c894659947ed5f8c9a93c60 2023-03-08 07:55:11,978 INFO [Listener at localhost/42999] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/.tmp/f1/0d74b0c61c53469299f0cb64bf21346c 2023-03-08 07:55:11,982 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/.tmp/f1/0d74b0c61c53469299f0cb64bf21346c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c 2023-03-08 07:55:11,986 INFO [Listener at localhost/42999] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 07:55:11,987 INFO [Listener at localhost/42999] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 240ad2a18a711975ba6b76a83e94017b in 830ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:11,987 DEBUG [Listener at localhost/42999] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-08 07:55:11,988 DEBUG [Listener at localhost/42999] regionserver.HRegion(2446): Flush status journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:11,990 INFO [Listener at localhost/42999] regionserver.HRegion(2745): Flushing 240ad2a18a711975ba6b76a83e94017b 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 07:55:12,320 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:55:12,401 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086eed0426aa754a09afab0bf16998c30a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303086eed0426aa754a09afab0bf16998c30a 2023-03-08 07:55:12,409 INFO [Listener at localhost/42999] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/.tmp/f1/1242f5f76c5f4058b4ecc2356685d999 2023-03-08 07:55:12,414 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/.tmp/f1/1242f5f76c5f4058b4ecc2356685d999 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999 2023-03-08 07:55:12,418 INFO [Listener at localhost/42999] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999, entries=1, sequenceid=9, filesize=4.9 K 2023-03-08 07:55:12,419 INFO [Listener at localhost/42999] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 240ad2a18a711975ba6b76a83e94017b in 429ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:12,420 DEBUG [Listener at localhost/42999] regionserver.HRegion(2446): Flush status journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:12,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-08 07:55:12,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 07:55:12,423 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 07:55:12,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:12,423 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262712423 2023-03-08 07:55:12,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 07:55:12,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-08 07:55:12,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-08 07:55:12,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x0482ed82 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:12,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ae00f7b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:12,430 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 07:55:12,473 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,473 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,524 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,524 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,575 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,575 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,626 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,626 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,677 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,677 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,728 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,728 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,778 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,779 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,829 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,829 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,880 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,880 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,931 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,931 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,982 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:12,982 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,033 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,033 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,084 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,084 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,135 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,135 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,186 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,186 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,237 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,237 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,255 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c6c80b09576040c78c815d736387e04e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308c6c80b09576040c78c815d736387e04e 2023-03-08 07:55:13,256 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 07:55:13,272 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:13,273 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52270, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:13,279 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:55:13,281 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59970, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:55:13,288 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,288 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,289 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/e689dd26a0ad47efbccf62ce726cfe92 first=Optional[r1] last=Optional[r2] 2023-03-08 07:55:13,290 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{f1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/e689dd26a0ad47efbccf62ce726cfe92}] 2023-03-08 07:55:13,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/e689dd26a0ad47efbccf62ce726cfe92 for inclusion in 240ad2a18a711975ba6b76a83e94017b/f1 2023-03-08 07:55:13,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 07:55:13,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:55:13,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:13,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/e689dd26a0ad47efbccf62ce726cfe92 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testGetAfterCompaction__o0a024gedvoteqonieoq6mkhldkevnfct273vo3opvlvjbnkm6n5kto3c0urufu5/f1/e689dd26a0ad47efbccf62ce726cfe92 2023-03-08 07:55:13,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testGetAfterCompaction__o0a024gedvoteqonieoq6mkhldkevnfct273vo3opvlvjbnkm6n5kto3c0urufu5/f1/e689dd26a0ad47efbccf62ce726cfe92 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_ 2023-03-08 07:55:13,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testGetAfterCompaction__o0a024gedvoteqonieoq6mkhldkevnfct273vo3opvlvjbnkm6n5kto3c0urufu5/f1/e689dd26a0ad47efbccf62ce726cfe92 into 240ad2a18a711975ba6b76a83e94017b/f1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_ - updating store file list. 2023-03-08 07:55:13,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_ into 240ad2a18a711975ba6b76a83e94017b/f1 2023-03-08 07:55:13,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testGetAfterCompaction__o0a024gedvoteqonieoq6mkhldkevnfct273vo3opvlvjbnkm6n5kto3c0urufu5/f1/e689dd26a0ad47efbccf62ce726cfe92 into 240ad2a18a711975ba6b76a83e94017b/f1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_) 2023-03-08 07:55:13,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testGetAfterCompaction__o0a024gedvoteqonieoq6mkhldkevnfct273vo3opvlvjbnkm6n5kto3c0urufu5/f1/e689dd26a0ad47efbccf62ce726cfe92 2023-03-08 07:55:13,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 240ad2a18a711975ba6b76a83e94017b/f1 is initiating minor compaction 2023-03-08 07:55:13,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@26caa73e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:13,303 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 240ad2a18a711975ba6b76a83e94017b/f1 in testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:13,303 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/.tmp, totalSize=10.0 K 2023-03-08 07:55:13,303 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 1242f5f76c5f4058b4ecc2356685d999, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:55:13,303 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 0a58d1513110498990f7c9bde5b953a9_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:55:13,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 07:55:13,305 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:13,307 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303086eed0426aa754a09afab0bf16998c30a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303086eed0426aa754a09afab0bf16998c30a 2023-03-08 07:55:13,308 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303089cf195061c894659947ed5f8c9a93c60 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303089cf195061c894659947ed5f8c9a93c60 2023-03-08 07:55:13,308 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-08 07:55:13,308 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 07:55:13,309 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 07:55:13,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-08 07:55:13,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:55:13,310 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 07:55:13,311 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 240ad2a18a711975ba6b76a83e94017b#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 76.92 MB/second 2023-03-08 07:55:13,312 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 888 msec 2023-03-08 07:55:13,339 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,339 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,339 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,343 DEBUG [Listener at localhost/42999] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 07:55:13,347 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52282, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 07:55:13,347 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33669] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:13,348 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-08 07:55:13,348 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 07:55:13,348 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33669] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 07:55:13,348 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33669] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. because compaction request was cancelled 2023-03-08 07:55:13,400 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,400 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,400 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,451 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,452 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,452 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,503 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,503 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,503 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,554 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,554 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,554 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,605 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,605 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,605 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,656 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,656 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,656 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,707 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,707 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,707 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,757 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,758 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,758 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,808 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,809 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,809 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,859 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,860 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,860 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,910 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,911 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,911 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,961 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:13,962 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:13,962 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,012 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:14,012 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,013 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,063 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:14,063 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,063 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,114 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:14,114 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,114 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,129 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/.tmp/f1/9382a3cd6267430b9c1224681f5af03c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/9382a3cd6267430b9c1224681f5af03c 2023-03-08 07:55:14,133 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 240ad2a18a711975ba6b76a83e94017b/f1 of 240ad2a18a711975ba6b76a83e94017b into 9382a3cd6267430b9c1224681f5af03c(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:55:14,133 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:14,133 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678262113302; duration=0sec 2023-03-08 07:55:14,134 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:14,165 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:14,165 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,165 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,165 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/9382a3cd6267430b9c1224681f5af03c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678262114125; access_time=1678262113308; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,216 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678262113250; access_time=1678262112442; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 07:55:14,216 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,216 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262112409; access_time=1678262111991; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,216 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/9382a3cd6267430b9c1224681f5af03c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678262114125; access_time=1678262113308; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_] to archive 2023-03-08 07:55:14,231 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:14,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/1242f5f76c5f4058b4ecc2356685d999 2023-03-08 07:55:14,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0a58d1513110498990f7c9bde5b953a9_SeqId_12_ 2023-03-08 07:55:14,267 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678262111977; access_time=1678262111159; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,267 INFO [Listener at localhost/42999] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/9382a3cd6267430b9c1224681f5af03c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678262114125; access_time=1678262113308; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 07:55:14,382 INFO [Listener at localhost/42999] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-08 07:55:14,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-08 07:55:14,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,395 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262114394"}]},"ts":"1678262114394"} 2023-03-08 07:55:14,396 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-08 07:55:14,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 07:55:14,397 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-08 07:55:14,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, UNASSIGN}] 2023-03-08 07:55:14,399 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, UNASSIGN 2023-03-08 07:55:14,400 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=240ad2a18a711975ba6b76a83e94017b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:14,400 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678262114400"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262114400"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262114400"}]},"ts":"1678262114400"} 2023-03-08 07:55:14,401 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 240ad2a18a711975ba6b76a83e94017b, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:14,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 07:55:14,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:14,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 240ad2a18a711975ba6b76a83e94017b, disabling compactions & flushes 2023-03-08 07:55:14,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:14,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:14,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. after waiting 0 ms 2023-03-08 07:55:14,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:14,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 07:55:14,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-08 07:55:14,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b. 2023-03-08 07:55:14,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 240ad2a18a711975ba6b76a83e94017b: 2023-03-08 07:55:14,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:14,560 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=240ad2a18a711975ba6b76a83e94017b, regionState=CLOSED 2023-03-08 07:55:14,560 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678262114560"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262114560"}]},"ts":"1678262114560"} 2023-03-08 07:55:14,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-08 07:55:14,563 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 240ad2a18a711975ba6b76a83e94017b, server=jenkins-hbase4.apache.org,33669,1678262016084 in 161 msec 2023-03-08 07:55:14,564 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-08 07:55:14,564 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=240ad2a18a711975ba6b76a83e94017b, UNASSIGN in 165 msec 2023-03-08 07:55:14,564 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262114564"}]},"ts":"1678262114564"} 2023-03-08 07:55:14,565 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-08 07:55:14,570 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-08 07:55:14,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 183 msec 2023-03-08 07:55:14,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 07:55:14,699 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-08 07:55:14,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-08 07:55:14,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,710 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,712 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 07:55:14,714 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:14,716 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1, FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/recovered.edits] 2023-03-08 07:55:14,719 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/0d74b0c61c53469299f0cb64bf21346c 2023-03-08 07:55:14,720 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/9382a3cd6267430b9c1224681f5af03c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/f1/9382a3cd6267430b9c1224681f5af03c 2023-03-08 07:55:14,723 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/recovered.edits/15.seqid to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b/recovered.edits/15.seqid 2023-03-08 07:55:14,723 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testGetAfterCompaction/240ad2a18a711975ba6b76a83e94017b 2023-03-08 07:55:14,723 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 07:55:14,723 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 07:55:14,724 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-08 07:55:14,727 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308c6c80b09576040c78c815d736387e04e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308c6c80b09576040c78c815d736387e04e 2023-03-08 07:55:14,727 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 07:55:14,729 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,752 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-08 07:55:14,754 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-08 07:55:14,755 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,756 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-08 07:55:14,756 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678262114756"}]},"ts":"9223372036854775807"} 2023-03-08 07:55:14,760 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-08 07:55:14,760 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 240ad2a18a711975ba6b76a83e94017b, NAME => 'testGetAfterCompaction,,1678262109039.240ad2a18a711975ba6b76a83e94017b.', STARTKEY => '', ENDKEY => ''}] 2023-03-08 07:55:14,760 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-08 07:55:14,760 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678262114760"}]},"ts":"9223372036854775807"} 2023-03-08 07:55:14,761 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-08 07:55:14,765 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 07:55:14,765 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 61 msec 2023-03-08 07:55:14,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 07:55:14,815 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-08 07:55:14,828 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=244 (was 240) - Thread LEAK? -, OpenFileDescriptor=622 (was 620) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=199 (was 217), ProcessCount=170 (was 170), AvailableMemoryMB=7554 (was 7555) 2023-03-08 07:55:14,839 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=244, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=199, ProcessCount=170, AvailableMemoryMB=7554 2023-03-08 07:55:14,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 07:55:14,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-08 07:55:14,844 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:55:14,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-08 07:55:14,844 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:55:14,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 07:55:14,847 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:14,847 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:14,847 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:14,847 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 empty. 2023-03-08 07:55:14,847 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e empty. 2023-03-08 07:55:14,848 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 empty. 2023-03-08 07:55:14,848 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:14,848 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:14,848 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:14,848 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-08 07:55:14,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 07:55:15,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 07:55:15,259 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 07:55:15,260 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => e9fb34d1dad0ba5071bdda7f618f683e, NAME => 'testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:15,260 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 2fb06f0faeeffea363aea80614ef1a51, NAME => 'testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:15,260 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => a50cf6d148fcdc856ce86b2cdca8ceb8, NAME => 'testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:15,274 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:15,274 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing e9fb34d1dad0ba5071bdda7f618f683e, disabling compactions & flushes 2023-03-08 07:55:15,275 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:15,275 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:15,275 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. after waiting 0 ms 2023-03-08 07:55:15,275 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:15,275 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:15,275 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:15,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 07:55:15,670 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:15,670 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing a50cf6d148fcdc856ce86b2cdca8ceb8, disabling compactions & flushes 2023-03-08 07:55:15,671 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:15,671 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:15,671 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. after waiting 0 ms 2023-03-08 07:55:15,671 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:15,671 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:15,671 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:15,677 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:15,677 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 2fb06f0faeeffea363aea80614ef1a51, disabling compactions & flushes 2023-03-08 07:55:15,677 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:15,677 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:15,677 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. after waiting 0 ms 2023-03-08 07:55:15,677 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:15,677 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:15,677 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:15,680 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:55:15,681 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262115681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262115681"}]},"ts":"1678262115681"} 2023-03-08 07:55:15,681 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262115681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262115681"}]},"ts":"1678262115681"} 2023-03-08 07:55:15,681 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262115681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262115681"}]},"ts":"1678262115681"} 2023-03-08 07:55:15,683 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 07:55:15,684 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:55:15,684 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262115684"}]},"ts":"1678262115684"} 2023-03-08 07:55:15,685 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 07:55:15,689 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN}] 2023-03-08 07:55:15,690 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN 2023-03-08 07:55:15,690 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN 2023-03-08 07:55:15,690 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN 2023-03-08 07:55:15,690 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:15,691 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:15,691 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:15,842 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2fb06f0faeeffea363aea80614ef1a51, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:15,842 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=a50cf6d148fcdc856ce86b2cdca8ceb8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:15,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=e9fb34d1dad0ba5071bdda7f618f683e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:15,842 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262115842"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262115842"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262115842"}]},"ts":"1678262115842"} 2023-03-08 07:55:15,842 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262115842"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262115842"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262115842"}]},"ts":"1678262115842"} 2023-03-08 07:55:15,842 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262115842"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262115842"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262115842"}]},"ts":"1678262115842"} 2023-03-08 07:55:15,844 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=81, state=RUNNABLE; OpenRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:15,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:15,846 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:15,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 07:55:15,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:15,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2fb06f0faeeffea363aea80614ef1a51, NAME => 'testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 07:55:16,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:16,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,001 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,002 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:16,002 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:16,002 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2fb06f0faeeffea363aea80614ef1a51 columnFamilyName family1 2023-03-08 07:55:16,003 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(310): Store=2fb06f0faeeffea363aea80614ef1a51/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:16,004 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,005 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2 2023-03-08 07:55:16,005 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2 2023-03-08 07:55:16,005 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2fb06f0faeeffea363aea80614ef1a51 columnFamilyName family2 2023-03-08 07:55:16,006 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(310): Store=2fb06f0faeeffea363aea80614ef1a51/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:16,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:16,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:16,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:16,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2fb06f0faeeffea363aea80614ef1a51; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70645414, jitterRate=0.05269870162010193}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:16,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:16,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51., pid=83, masterSystemTime=1678262115997 2023-03-08 07:55:16,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:16,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:16,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:16,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e9fb34d1dad0ba5071bdda7f618f683e, NAME => 'testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 07:55:16,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2fb06f0faeeffea363aea80614ef1a51, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:16,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:16,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262116013"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262116013"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262116013"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262116013"}]},"ts":"1678262116013"} 2023-03-08 07:55:16,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,014 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,015 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:16,015 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:16,016 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9fb34d1dad0ba5071bdda7f618f683e columnFamilyName family1 2023-03-08 07:55:16,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=81 2023-03-08 07:55:16,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=81, state=SUCCESS; OpenRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084 in 171 msec 2023-03-08 07:55:16,017 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(310): Store=e9fb34d1dad0ba5071bdda7f618f683e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:16,017 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN in 329 msec 2023-03-08 07:55:16,017 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,018 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2 2023-03-08 07:55:16,018 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2 2023-03-08 07:55:16,018 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9fb34d1dad0ba5071bdda7f618f683e columnFamilyName family2 2023-03-08 07:55:16,019 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(310): Store=e9fb34d1dad0ba5071bdda7f618f683e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:16,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:16,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:16,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:16,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e9fb34d1dad0ba5071bdda7f618f683e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60858385, jitterRate=-0.0931393951177597}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:16,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:16,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e., pid=84, masterSystemTime=1678262115997 2023-03-08 07:55:16,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:16,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:16,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:16,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a50cf6d148fcdc856ce86b2cdca8ceb8, NAME => 'testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 07:55:16,028 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=e9fb34d1dad0ba5071bdda7f618f683e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:16,028 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262116028"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262116028"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262116028"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262116028"}]},"ts":"1678262116028"} 2023-03-08 07:55:16,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:16,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,030 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,031 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-03-08 07:55:16,031 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084 in 185 msec 2023-03-08 07:55:16,031 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:16,031 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:16,032 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a50cf6d148fcdc856ce86b2cdca8ceb8 columnFamilyName family1 2023-03-08 07:55:16,032 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN in 342 msec 2023-03-08 07:55:16,032 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(310): Store=a50cf6d148fcdc856ce86b2cdca8ceb8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:16,033 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,034 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2 2023-03-08 07:55:16,034 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2 2023-03-08 07:55:16,034 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a50cf6d148fcdc856ce86b2cdca8ceb8 columnFamilyName family2 2023-03-08 07:55:16,035 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(310): Store=a50cf6d148fcdc856ce86b2cdca8ceb8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:16,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:16,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:16,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:16,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a50cf6d148fcdc856ce86b2cdca8ceb8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70404904, jitterRate=0.04911482334136963}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:16,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:16,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8., pid=85, masterSystemTime=1678262115997 2023-03-08 07:55:16,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:16,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:16,043 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=a50cf6d148fcdc856ce86b2cdca8ceb8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:16,043 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262116043"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262116043"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262116043"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262116043"}]},"ts":"1678262116043"} 2023-03-08 07:55:16,046 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-03-08 07:55:16,046 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 199 msec 2023-03-08 07:55:16,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-08 07:55:16,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN in 359 msec 2023-03-08 07:55:16,047 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:55:16,048 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262116048"}]},"ts":"1678262116048"} 2023-03-08 07:55:16,049 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 07:55:16,051 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:55:16,052 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2090 sec 2023-03-08 07:55:16,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 07:55:16,950 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-08 07:55:16,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:16,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:16,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:16,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:16,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 07:55:16,964 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 07:55:16,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:16,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 07:55:16,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 07:55:16,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:16,964 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:16,971 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:16,971 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:16,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:16,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:16,972 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:16,972 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:16,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:16,972 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:16,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:16,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:16,973 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 07:55:16,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 07:55:16,973 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 07:55:16,973 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:16,973 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 07:55:16,973 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:16,973 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. started... 2023-03-08 07:55:16,973 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:16,973 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:16,974 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. started... 2023-03-08 07:55:16,974 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2fb06f0faeeffea363aea80614ef1a51 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:55:16,973 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:16,974 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e9fb34d1dad0ba5071bdda7f618f683e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:55:16,973 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:16,974 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. started... 2023-03-08 07:55:16,975 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a50cf6d148fcdc856ce86b2cdca8ceb8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:55:17,392 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe 2023-03-08 07:55:17,392 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a 2023-03-08 07:55:17,393 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 2023-03-08 07:55:17,398 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/8986c97ff59f4b56bf600bf2a1520ee7 2023-03-08 07:55:17,398 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/04fda751d9ed4b71bdfbd09e53c14b7b 2023-03-08 07:55:17,799 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/b1a0e74efa4c4846bc9024a6de7e8074 2023-03-08 07:55:17,814 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b000d47aaa2d4ff4a3800fc3e2b0c824 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b000d47aaa2d4ff4a3800fc3e2b0c824 2023-03-08 07:55:17,814 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030897dbef609cad42e29e296927c255dd6f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030897dbef609cad42e29e296927c255dd6f 2023-03-08 07:55:17,820 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/f46ce0a8709142a28bb3626c172a4a46 2023-03-08 07:55:17,823 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/8986c97ff59f4b56bf600bf2a1520ee7 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7 2023-03-08 07:55:17,827 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 07:55:17,827 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/f46ce0a8709142a28bb3626c172a4a46 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46 2023-03-08 07:55:17,830 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:55:17,831 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a50cf6d148fcdc856ce86b2cdca8ceb8 in 856ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:17,831 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 07:55:17,831 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:17,831 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:17,832 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:18,212 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d07ea5cdca74445f824b90fa36aca6ef to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d07ea5cdca74445f824b90fa36aca6ef 2023-03-08 07:55:18,220 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/9cae59daa977464dbc7c2d8b8a2bed4a 2023-03-08 07:55:18,224 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/04fda751d9ed4b71bdfbd09e53c14b7b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b 2023-03-08 07:55:18,228 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 07:55:18,229 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/9cae59daa977464dbc7c2d8b8a2bed4a as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a 2023-03-08 07:55:18,235 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:55:18,236 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e9fb34d1dad0ba5071bdda7f618f683e in 1262ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:18,237 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:18,237 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:18,237 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:18,617 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/650c0bf692e9431184c7ef97d0ef75ea 2023-03-08 07:55:18,622 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/b1a0e74efa4c4846bc9024a6de7e8074 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074 2023-03-08 07:55:18,625 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 07:55:18,625 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/650c0bf692e9431184c7ef97d0ef75ea as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea 2023-03-08 07:55:18,629 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:55:18,630 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 2fb06f0faeeffea363aea80614ef1a51 in 1656ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:18,630 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:18,630 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:18,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:18,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:18,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 07:55:18,630 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 07:55:18,633 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,633 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:18,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:18,633 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,633 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:18,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:18,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:18,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:18,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:18,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 07:55:18,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a686580[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:18,634 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 07:55:18,634 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,636 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,636 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 07:55:18,636 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 07:55:18,636 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:18,636 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,636 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:18,638 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 07:55:18,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:18,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:18,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:18,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 07:55:18,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:18,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:18,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:18,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:18,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:18,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:18,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 07:55:18,640 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 07:55:18,640 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:18,640 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:18,640 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 07:55:18,640 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:18,642 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,642 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,642 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:18,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:18,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:18,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:18,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:18,643 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,644 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,646 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,646 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:18,646 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:18,646 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:18,646 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:18,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:18,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:18,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 07:55:18,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 07:55:18,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:18,646 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:18,647 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,647 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,647 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:18,647 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:18,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:18,647 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:18,647 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:18,747 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:18,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:18,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:18,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:18,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:18,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:18,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:18,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 07:55:18,764 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 07:55:18,764 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:18,764 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 07:55:18,764 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 07:55:18,764 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,765 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:18,766 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:18,766 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:18,766 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:18,766 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:18,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:18,767 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 07:55:18,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 07:55:18,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 07:55:18,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 07:55:18,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:18,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 07:55:18,768 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:18,768 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. started... 2023-03-08 07:55:18,768 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:18,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:18,768 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2fb06f0faeeffea363aea80614ef1a51 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:55:18,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:18,768 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:18,769 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. started... 2023-03-08 07:55:18,768 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. started... 2023-03-08 07:55:18,769 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e9fb34d1dad0ba5071bdda7f618f683e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:55:18,769 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a50cf6d148fcdc856ce86b2cdca8ceb8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:55:19,198 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 2023-03-08 07:55:19,198 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb 2023-03-08 07:55:19,199 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b 2023-03-08 07:55:19,203 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/32ba1adf9a024898b96e4d8e4abf926c 2023-03-08 07:55:19,604 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/deb2ab70953849c584ea0ec88bf3ed81 2023-03-08 07:55:19,604 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/4d800d09acb843c68c40907b9749c4b0 2023-03-08 07:55:19,616 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b901881636af44f69d119f6558b2ab98 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b901881636af44f69d119f6558b2ab98 2023-03-08 07:55:19,618 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308bc16c75ca33c4bf7bae11cc42bc1c2a8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308bc16c75ca33c4bf7bae11cc42bc1c2a8 2023-03-08 07:55:19,625 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/7e24d05a9202460e924485dec0ce3271 2023-03-08 07:55:19,629 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/32ba1adf9a024898b96e4d8e4abf926c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c 2023-03-08 07:55:19,633 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 07:55:19,633 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/7e24d05a9202460e924485dec0ce3271 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271 2023-03-08 07:55:19,636 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:55:19,637 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e9fb34d1dad0ba5071bdda7f618f683e in 868ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:19,637 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:19,637 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:19,637 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:20,019 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d9ea1e6fd03e4340b7e67abe1a346544 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d9ea1e6fd03e4340b7e67abe1a346544 2023-03-08 07:55:20,027 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/bec62040e49a41f3892006664123aba7 2023-03-08 07:55:20,031 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/4d800d09acb843c68c40907b9749c4b0 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0 2023-03-08 07:55:20,034 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 07:55:20,034 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/bec62040e49a41f3892006664123aba7 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7 2023-03-08 07:55:20,038 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:55:20,038 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 2fb06f0faeeffea363aea80614ef1a51 in 1270ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:20,039 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:20,039 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:20,039 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:20,060 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:55:20,427 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/7ff98be37e5643c59927680a6da57fdc 2023-03-08 07:55:20,431 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/deb2ab70953849c584ea0ec88bf3ed81 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81 2023-03-08 07:55:20,435 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 07:55:20,435 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/7ff98be37e5643c59927680a6da57fdc as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc 2023-03-08 07:55:20,439 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:55:20,440 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a50cf6d148fcdc856ce86b2cdca8ceb8 in 1671ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:20,440 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:20,440 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 07:55:20,440 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 07:55:20,441 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,441 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:20,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:20,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,442 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:20,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:20,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:20,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:20,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 07:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a021023[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:20,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 07:55:20,443 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,450 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,450 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 07:55:20,450 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 07:55:20,450 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:20,450 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,450 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:20,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 07:55:20,451 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:20,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:20,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:20,451 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 07:55:20,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:20,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:20,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:20,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:20,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:20,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:20,453 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 07:55:20,453 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 07:55:20,453 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:20,453 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:20,453 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 07:55:20,453 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:20,455 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,455 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,455 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:20,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:20,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:20,455 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:20,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:20,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:20,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:20,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:20,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:20,456 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:20,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:20,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,458 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:20,458 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:20,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:20,458 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:20,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 07:55:20,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:20,458 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:20,458 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:20,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:20,459 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:20,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:20,559 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:20,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:20,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:20,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:20,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:20,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:20,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:20,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 07:55:20,566 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 07:55:20,567 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:20,567 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 07:55:20,567 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 07:55:20,567 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,567 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:20,574 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,574 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:20,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:20,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:20,575 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:20,575 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:20,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:20,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:20,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:20,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:20,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 07:55:20,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 07:55:20,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 07:55:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 07:55:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 07:55:20,576 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:20,576 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. started... 2023-03-08 07:55:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:20,576 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:20,576 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:20,576 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. started... 2023-03-08 07:55:20,576 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. started... 2023-03-08 07:55:20,576 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2fb06f0faeeffea363aea80614ef1a51 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:55:20,576 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a50cf6d148fcdc856ce86b2cdca8ceb8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:55:20,576 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:20,576 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e9fb34d1dad0ba5071bdda7f618f683e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:55:20,624 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 2023-03-08 07:55:21,009 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 2023-03-08 07:55:21,017 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 2023-03-08 07:55:21,030 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/f32fa60784ae4ddabafe1a8293029119 2023-03-08 07:55:21,414 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/f8a89224d8404fd3b8cec9768b68df23 2023-03-08 07:55:21,421 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/5e9ed625067b4b08a3f7b952e348a0b7 2023-03-08 07:55:21,441 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308fcbe739713b745cab5958eadbc0dbd42 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308fcbe739713b745cab5958eadbc0dbd42 2023-03-08 07:55:21,829 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086439fd156cdc46e484e3bdd40ca1b107 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303086439fd156cdc46e484e3bdd40ca1b107 2023-03-08 07:55:21,834 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/c33d6be82d7e460db1fdbbf7131c6545 2023-03-08 07:55:21,835 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088972274e07814bcb8afc2d88bbee21e2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303088972274e07814bcb8afc2d88bbee21e2 2023-03-08 07:55:21,838 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/f8a89224d8404fd3b8cec9768b68df23 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23 2023-03-08 07:55:21,840 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/6f57e6ccdff045cd8423070fb969e229 2023-03-08 07:55:21,842 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 07:55:21,843 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/c33d6be82d7e460db1fdbbf7131c6545 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545 2023-03-08 07:55:21,844 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/5e9ed625067b4b08a3f7b952e348a0b7 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7 2023-03-08 07:55:21,845 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/b6c0e39cb8fd4b39a6570ccd11d3b508 2023-03-08 07:55:21,847 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:55:21,847 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 07:55:21,848 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a50cf6d148fcdc856ce86b2cdca8ceb8 in 1272ms, sequenceid=13, compaction requested=true 2023-03-08 07:55:21,848 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:21,848 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:21,848 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/6f57e6ccdff045cd8423070fb969e229 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229 2023-03-08 07:55:21,850 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/f32fa60784ae4ddabafe1a8293029119 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119 2023-03-08 07:55:21,852 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:55:21,853 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 2fb06f0faeeffea363aea80614ef1a51 in 1277ms, sequenceid=13, compaction requested=true 2023-03-08 07:55:21,853 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:21,853 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:21,853 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:21,853 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 07:55:21,854 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/b6c0e39cb8fd4b39a6570ccd11d3b508 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508 2023-03-08 07:55:21,857 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:55:21,857 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e9fb34d1dad0ba5071bdda7f618f683e in 1281ms, sequenceid=13, compaction requested=true 2023-03-08 07:55:21,857 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:21,857 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:21,857 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:21,857 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:21,858 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,858 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 07:55:21,858 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 07:55:21,859 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,859 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:21,859 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,859 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:21,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 07:55:21,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f829ccf[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:21,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 07:55:21,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,862 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,862 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 07:55:21,862 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 07:55:21,862 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,862 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:21,862 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:21,863 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,863 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 07:55:21,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:21,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:21,863 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:21,863 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 07:55:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:21,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 07:55:21,866 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 07:55:21,866 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:21,866 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:21,866 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 07:55:21,866 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:21,872 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,872 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,872 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:21,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:21,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:21,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,873 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:21,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:21,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:21,874 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:21,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:21,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:21,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:21,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 07:55:21,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:21,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:21,876 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:21,876 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,976 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:21,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:21,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:21,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:21,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:21,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:21,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:21,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 07:55:21,984 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 07:55:21,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:21,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 07:55:21,984 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 07:55:21,985 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,985 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:21,986 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,986 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:21,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:21,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:21,986 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:21,986 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:21,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:21,987 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:21,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:21,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:21,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 07:55:21,988 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 07:55:21,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 07:55:21,988 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 07:55:21,988 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:21,988 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 07:55:21,988 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:21,988 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:21,988 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:21,988 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:21,988 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. started... 2023-03-08 07:55:21,988 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. started... 2023-03-08 07:55:21,988 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:21,988 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e9fb34d1dad0ba5071bdda7f618f683e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:55:21,988 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. started... 2023-03-08 07:55:21,988 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a50cf6d148fcdc856ce86b2cdca8ceb8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:55:21,989 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2fb06f0faeeffea363aea80614ef1a51 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:55:22,417 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b 2023-03-08 07:55:22,417 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 2023-03-08 07:55:22,417 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 2023-03-08 07:55:22,424 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc 2023-03-08 07:55:22,822 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/57716fd64aec4b8a9f14ced6989e1725 2023-03-08 07:55:22,824 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/c15382d6134c492ab93f119690918ab1 2023-03-08 07:55:22,843 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308db8d12fe113142e58b12a3aa4db49c42 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308db8d12fe113142e58b12a3aa4db49c42 2023-03-08 07:55:22,850 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/9c31cc68bf324de28df80734eb1c30e2 2023-03-08 07:55:22,852 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308dd902299661f4b9abddc12dccc114e5b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd902299661f4b9abddc12dccc114e5b 2023-03-08 07:55:22,854 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc 2023-03-08 07:55:22,858 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 07:55:22,858 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/9c31cc68bf324de28df80734eb1c30e2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2 2023-03-08 07:55:22,862 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:55:22,862 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e9fb34d1dad0ba5071bdda7f618f683e in 874ms, sequenceid=17, compaction requested=true 2023-03-08 07:55:22,862 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:22,862 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:22,862 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:23,243 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030894dc8b9e31a3439c9805e0e258f462e7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030894dc8b9e31a3439c9805e0e258f462e7 2023-03-08 07:55:23,257 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/ae1cf04043de49c884f5504f54ee4297 2023-03-08 07:55:23,260 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/c15382d6134c492ab93f119690918ab1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1 2023-03-08 07:55:23,263 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 07:55:23,264 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/ae1cf04043de49c884f5504f54ee4297 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297 2023-03-08 07:55:23,267 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:55:23,268 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 2fb06f0faeeffea363aea80614ef1a51 in 1279ms, sequenceid=17, compaction requested=true 2023-03-08 07:55:23,268 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:23,268 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:23,268 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:23,648 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/4b19bbe0117d46e4937560e8cb137e61 2023-03-08 07:55:23,653 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/57716fd64aec4b8a9f14ced6989e1725 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725 2023-03-08 07:55:23,656 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 07:55:23,656 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/4b19bbe0117d46e4937560e8cb137e61 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61 2023-03-08 07:55:23,660 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:55:23,661 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a50cf6d148fcdc856ce86b2cdca8ceb8 in 1673ms, sequenceid=17, compaction requested=true 2023-03-08 07:55:23,661 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:23,661 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 07:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 07:55:23,663 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,663 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:23,663 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,663 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:23,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:23,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:23,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:23,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 07:55:23,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@492dc5c[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:23,665 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 07:55:23,665 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,667 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,667 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 07:55:23,667 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 07:55:23,667 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:23,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,667 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 07:55:23,668 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 07:55:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:23,671 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 07:55:23,671 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:23,671 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:23,671 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 07:55:23,671 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:23,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 07:55:23,672 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,672 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,672 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:23,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:23,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:23,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:23,673 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:23,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:23,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:23,674 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:23,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:23,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,676 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:23,676 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:23,676 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:23,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 07:55:23,676 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:23,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 07:55:23,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:23,676 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:23,676 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:23,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:23,676 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:23,677 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:23,677 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:23,677 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,677 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:23,677 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:23,777 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:23,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:23,788 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:23,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678262123788 (current time:1678262123788). 2023-03-08 07:55:23,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 07:55:23,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] zookeeper.ReadOnlyZKClient(139): Connect 0x5074f24f to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:23,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@612314c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:23,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:23,803 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59160, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:23,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5074f24f to 127.0.0.1:52387 2023-03-08 07:55:23,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:23,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 07:55:23,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:23,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 07:55:23,813 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 07:55:23,814 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678262723814 2023-03-08 07:55:23,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:23,814 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678262123778 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 07:55:23,815 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 07:55:23,816 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 07:55:23,816 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-03-08 07:55:23,816 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 07:55:23,817 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678262723817 2023-03-08 07:55:23,819 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:23,819 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 07:55:23,919 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:23,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:23,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:23,923 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 07:55:24,123 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:24,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:24,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:24,125 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 07:55:24,225 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678262123778 2023-03-08 07:55:24,225 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678262123778' 2023-03-08 07:55:24,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 07:55:24,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678262123778' starting 'acquire' 2023-03-08 07:55:24,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678262123778', kicking off acquire phase on members. 2023-03-08 07:55:24,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678262123778 2023-03-08 07:55:24,226 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 07:55:24,226 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 07:55:24,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 07:55:24,227 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,227 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:24,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678262123778 2023-03-08 07:55:24,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-08 07:55:24,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678262123778 2023-03-08 07:55:24,228 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678262123778 from table testCompactionWithHFileLink type FLUSH 2023-03-08 07:55:24,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678262123778 2023-03-08 07:55:24,231 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678262123778' with timeout 300000ms 2023-03-08 07:55:24,231 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 07:55:24,231 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678262123778' starting 'acquire' stage 2023-03-08 07:55:24,231 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678262123778' locally acquired 2023-03-08 07:55:24,231 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (snaptb-1678262123778) in zk 2023-03-08 07:55:24,232 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,232 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:24,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 07:55:24,233 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,233 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678262123778' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:24,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:24,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:24,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678262123778 2023-03-08 07:55:24,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:24,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'snaptb-1678262123778' on coordinator 2023-03-08 07:55:24,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d8ffd2b[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:24,234 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678262123778' starting 'in-barrier' execution. 2023-03-08 07:55:24,234 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,238 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,238 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:24,238 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678262123778' received 'reached' from coordinator. 2023-03-08 07:55:24,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:24,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-08 07:55:24,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:24,239 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-08 07:55:24,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:24,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. started... 2023-03-08 07:55:24,239 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. started... 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. started... 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.' region-info for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.' region-info for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,240 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.' region-info for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,246 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 07:55:24,246 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 07:55:24,246 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 07:55:24,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1] hfiles 2023-03-08 07:55:24,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725] hfiles 2023-03-08 07:55:24,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc] hfiles 2023-03-08 07:55:24,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,434 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:24,435 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:24,435 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-08 07:55:24,435 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0482ed82 to 127.0.0.1:52387 2023-03-08 07:55:24,436 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:24,436 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0526bd06 to 127.0.0.1:52387 2023-03-08 07:55:24,436 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:24,436 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:24,436 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2110ec23 to 127.0.0.1:52387 2023-03-08 07:55:24,436 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:24,436 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:24,436 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10a2a106 to 127.0.0.1:52387 2023-03-08 07:55:24,436 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:24,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:24,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:24,437 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2] hfiles 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61] hfiles 2023-03-08 07:55:24,437 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297] hfiles 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,438 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2 for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,448 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. completed. 2023-03-08 07:55:24,448 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:24,449 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-08 07:55:24,845 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. completed. 2023-03-08 07:55:24,845 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:24,845 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. completed. 2023-03-08 07:55:24,845 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-08 07:55:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-08 07:55:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-08 07:55:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678262123778' locally completed 2023-03-08 07:55:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678262123778' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678262123778' has notified controller of completion 2023-03-08 07:55:24,847 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 07:55:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678262123778' completed. 2023-03-08 07:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678262123778 2023-03-08 07:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678262123778 2023-03-08 07:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678262123778' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'snaptb-1678262123778', counting down latch. Waiting for 0 more 2023-03-08 07:55:24,849 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678262123778' execution completed 2023-03-08 07:55:24,850 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:24,850 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:24,850 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678262123778 2023-03-08 07:55:24,850 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678262123778including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-08 07:55:24,851 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,851 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 07:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,851 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 07:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:24,852 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 07:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 07:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678262123778 2023-03-08 07:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678262123778 2023-03-08 07:55:24,853 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678262123778 2023-03-08 07:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,855 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,855 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 07:55:24,855 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:24,855 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678262123778 2023-03-08 07:55:24,855 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678262123778 2023-03-08 07:55:24,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 07:55:24,855 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 07:55:24,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 07:55:24,855 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-08 07:55:24,855 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678262123778 2023-03-08 07:55:24,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1678262123778 because node does not exist (not an error) 2023-03-08 07:55:24,856 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678262123778 2023-03-08 07:55:24,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 07:55:24,856 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-08 07:55:24,856 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678262123778/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:24,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 07:55:24,856 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,856 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678262123778 2023-03-08 07:55:24,856 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678262123778 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b] mob files 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 2023-03-08 07:55:24,858 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303088972274e07814bcb8afc2d88bbee21e2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308bc16c75ca33c4bf7bae11cc42bc1c2a8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d07ea5cdca74445f824b90fa36aca6ef, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd902299661f4b9abddc12dccc114e5b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030897dbef609cad42e29e296927c255dd6f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b901881636af44f69d119f6558b2ab98, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308db8d12fe113142e58b12a3aa4db49c42, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308fcbe739713b745cab5958eadbc0dbd42, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303086439fd156cdc46e484e3bdd40ca1b107, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030894dc8b9e31a3439c9805e0e258f462e7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b000d47aaa2d4ff4a3800fc3e2b0c824, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d9ea1e6fd03e4340b7e67abe1a346544] mob files 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303088972274e07814bcb8afc2d88bbee21e2 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308bc16c75ca33c4bf7bae11cc42bc1c2a8 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d07ea5cdca74445f824b90fa36aca6ef 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd902299661f4b9abddc12dccc114e5b 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030897dbef609cad42e29e296927c255dd6f 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b901881636af44f69d119f6558b2ab98 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308db8d12fe113142e58b12a3aa4db49c42 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308fcbe739713b745cab5958eadbc0dbd42 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303086439fd156cdc46e484e3bdd40ca1b107 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030894dc8b9e31a3439c9805e0e258f462e7 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b000d47aaa2d4ff4a3800fc3e2b0c824 2023-03-08 07:55:24,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d9ea1e6fd03e4340b7e67abe1a346544 2023-03-08 07:55:24,937 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:24,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:24,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:24,938 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 07:55:25,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678262123778 2023-03-08 07:55:25,266 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/snaptb-1678262123778 2023-03-08 07:55:25,716 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/snaptb-1678262123778 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/snaptb-1678262123778 2023-03-08 07:55:25,738 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678262123778 of table testCompactionWithHFileLink completed 2023-03-08 07:55:25,738 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/snaptb-1678262123778 2023-03-08 07:55:25,739 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/snaptb-1678262123778 2023-03-08 07:55:25,739 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678262123778 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678262123814Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678262125264 (+1450 ms)Consolidate snapshot: snaptb-1678262123778 at 1678262125264Loading Region manifests for snaptb-1678262123778 at 1678262125265 (+1 ms)Writing data manifest for snaptb-1678262123778 at 1678262125274 (+9 ms)Verifying snapshot: snaptb-1678262123778 at 1678262125680 (+406 ms)Snapshot snaptb-1678262123778 of table testCompactionWithHFileLink completed at 1678262125738 (+58 ms) 2023-03-08 07:55:25,740 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 07:55:25,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.9250 sec 2023-03-08 07:55:25,938 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:25,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:25,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 07:55:25,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:25,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:25,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 07:55:25,972 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 07:55:25,972 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:25,972 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 07:55:25,972 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 07:55:25,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:25,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:25,974 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:25,974 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:25,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:25,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:25,974 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:25,974 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:25,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:25,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:25,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:25,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:25,975 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 07:55:25,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 07:55:25,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 07:55:25,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:25,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 07:55:25,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:25,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. started... 2023-03-08 07:55:25,976 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2fb06f0faeeffea363aea80614ef1a51 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:55:25,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:25,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:25,976 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:25,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:25,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. started... 2023-03-08 07:55:25,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. started... 2023-03-08 07:55:25,976 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e9fb34d1dad0ba5071bdda7f618f683e 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:55:25,977 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a50cf6d148fcdc856ce86b2cdca8ceb8 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:55:26,399 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/9ace12dd5b7e49f89be217abd1c058c4 2023-03-08 07:55:26,402 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ace12dd5b7e49f89be217abd1c058c4 2023-03-08 07:55:26,798 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/b0c84436b1c447429244807dd62c8e4f 2023-03-08 07:55:26,800 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/6379d180c1c74c4ab36bc3e0f489e814 2023-03-08 07:55:26,801 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b0c84436b1c447429244807dd62c8e4f 2023-03-08 07:55:26,803 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6379d180c1c74c4ab36bc3e0f489e814 2023-03-08 07:55:26,824 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/930c31d1eb7c4b92a609178381b469ea 2023-03-08 07:55:26,826 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/c26c3c0a78f146899623a949f7744f3f 2023-03-08 07:55:26,828 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 930c31d1eb7c4b92a609178381b469ea 2023-03-08 07:55:26,829 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/9ace12dd5b7e49f89be217abd1c058c4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/9ace12dd5b7e49f89be217abd1c058c4 2023-03-08 07:55:26,830 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c26c3c0a78f146899623a949f7744f3f 2023-03-08 07:55:26,831 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/b0c84436b1c447429244807dd62c8e4f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/b0c84436b1c447429244807dd62c8e4f 2023-03-08 07:55:26,833 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ace12dd5b7e49f89be217abd1c058c4 2023-03-08 07:55:26,833 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/9ace12dd5b7e49f89be217abd1c058c4, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 07:55:26,834 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/930c31d1eb7c4b92a609178381b469ea as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/930c31d1eb7c4b92a609178381b469ea 2023-03-08 07:55:26,835 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b0c84436b1c447429244807dd62c8e4f 2023-03-08 07:55:26,835 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/b0c84436b1c447429244807dd62c8e4f, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 07:55:26,836 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/c26c3c0a78f146899623a949f7744f3f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/c26c3c0a78f146899623a949f7744f3f 2023-03-08 07:55:26,839 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 930c31d1eb7c4b92a609178381b469ea 2023-03-08 07:55:26,839 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/930c31d1eb7c4b92a609178381b469ea, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 07:55:26,840 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 2fb06f0faeeffea363aea80614ef1a51 in 864ms, sequenceid=24, compaction requested=true 2023-03-08 07:55:26,840 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:26,840 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:26,840 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:26,840 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c26c3c0a78f146899623a949f7744f3f 2023-03-08 07:55:26,840 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/c26c3c0a78f146899623a949f7744f3f, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 07:55:26,841 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e9fb34d1dad0ba5071bdda7f618f683e in 865ms, sequenceid=24, compaction requested=true 2023-03-08 07:55:26,841 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:26,841 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:26,841 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:27,625 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/e959ef0924604bb58c6c998337cdd24f 2023-03-08 07:55:27,629 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e959ef0924604bb58c6c998337cdd24f 2023-03-08 07:55:27,629 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/6379d180c1c74c4ab36bc3e0f489e814 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/6379d180c1c74c4ab36bc3e0f489e814 2023-03-08 07:55:27,633 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6379d180c1c74c4ab36bc3e0f489e814 2023-03-08 07:55:27,633 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/6379d180c1c74c4ab36bc3e0f489e814, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 07:55:27,634 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/e959ef0924604bb58c6c998337cdd24f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/e959ef0924604bb58c6c998337cdd24f 2023-03-08 07:55:27,637 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e959ef0924604bb58c6c998337cdd24f 2023-03-08 07:55:27,637 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/e959ef0924604bb58c6c998337cdd24f, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 07:55:27,638 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a50cf6d148fcdc856ce86b2cdca8ceb8 in 1661ms, sequenceid=24, compaction requested=true 2023-03-08 07:55:27,638 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:27,638 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:27,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:27,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:27,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 07:55:27,638 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 07:55:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,640 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:27,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 07:55:27,641 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 07:55:27,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47967db1[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:27,641 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,646 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 07:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 07:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:27,647 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,647 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:27,648 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,648 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 07:55:27,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:27,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:27,648 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:27,648 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 07:55:27,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 07:55:27,650 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 07:55:27,650 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:27,650 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:27,650 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 07:55:27,650 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:27,651 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,652 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:27,652 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:27,652 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:27,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,654 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:27,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 07:55:27,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,656 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,656 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:27,656 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:27,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:27,656 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 07:55:27,656 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:27,657 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:27,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 07:55:27,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:27,657 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:27,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:27,657 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,657 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 07:55:27,657 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 07:55:27,657 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:27,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:27,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:27,657 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:27,757 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:27,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:27,759 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:27,759 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 2fb06f0faeeffea363aea80614ef1a51/family1 is initiating major compaction (all files) 2023-03-08 07:55:27,759 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:27,759 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:27,759 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 2fb06f0faeeffea363aea80614ef1a51/family1 in testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:27,760 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/9ace12dd5b7e49f89be217abd1c058c4] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp, totalSize=26.7 K 2023-03-08 07:55:27,760 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b1a0e74efa4c4846bc9024a6de7e8074, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116958 2023-03-08 07:55:27,761 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 4d800d09acb843c68c40907b9749c4b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:27,761 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 5e9ed625067b4b08a3f7b952e348a0b7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:27,762 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting c15382d6134c492ab93f119690918ab1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:27,762 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 9ace12dd5b7e49f89be217abd1c058c4, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 07:55:27,776 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 2fb06f0faeeffea363aea80614ef1a51#family1#compaction#107 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:28,184 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del 2023-03-08 07:55:28,185 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del 2023-03-08 07:55:28,998 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 739644265f1e4f6b899269a64eed8f18 2023-03-08 07:55:28,999 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/739644265f1e4f6b899269a64eed8f18 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/739644265f1e4f6b899269a64eed8f18 2023-03-08 07:55:29,002 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 739644265f1e4f6b899269a64eed8f18 2023-03-08 07:55:29,003 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2fb06f0faeeffea363aea80614ef1a51/family1 of 2fb06f0faeeffea363aea80614ef1a51 into 739644265f1e4f6b899269a64eed8f18(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:29,003 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:29,003 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:29,003 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 2fb06f0faeeffea363aea80614ef1a51/family2 is initiating major compaction (all files) 2023-03-08 07:55:29,003 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:29,003 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:29,003 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 2fb06f0faeeffea363aea80614ef1a51/family2 in testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:29,004 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/930c31d1eb7c4b92a609178381b469ea] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp, totalSize=25.4 K 2023-03-08 07:55:29,004 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 650c0bf692e9431184c7ef97d0ef75ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116958 2023-03-08 07:55:29,004 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting bec62040e49a41f3892006664123aba7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:29,004 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 6f57e6ccdff045cd8423070fb969e229, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:29,005 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting ae1cf04043de49c884f5504f54ee4297, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:29,005 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 930c31d1eb7c4b92a609178381b469ea, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 07:55:29,016 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 2fb06f0faeeffea363aea80614ef1a51#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:29,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/9ace12dd5b7e49f89be217abd1c058c4] to archive 2023-03-08 07:55:29,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:29,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/b1a0e74efa4c4846bc9024a6de7e8074 2023-03-08 07:55:29,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/4d800d09acb843c68c40907b9749c4b0 2023-03-08 07:55:29,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/5e9ed625067b4b08a3f7b952e348a0b7 2023-03-08 07:55:29,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/c15382d6134c492ab93f119690918ab1 2023-03-08 07:55:29,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/9ace12dd5b7e49f89be217abd1c058c4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/9ace12dd5b7e49f89be217abd1c058c4 2023-03-08 07:55:29,426 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del 2023-03-08 07:55:29,426 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del 2023-03-08 07:55:30,239 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3178b0499df84c24b7261c416aa4b750 2023-03-08 07:55:30,240 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/3178b0499df84c24b7261c416aa4b750 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/3178b0499df84c24b7261c416aa4b750 2023-03-08 07:55:30,244 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3178b0499df84c24b7261c416aa4b750 2023-03-08 07:55:30,245 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2fb06f0faeeffea363aea80614ef1a51/family2 of 2fb06f0faeeffea363aea80614ef1a51 into 3178b0499df84c24b7261c416aa4b750(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:30,245 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:30,245 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:30,245 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): a50cf6d148fcdc856ce86b2cdca8ceb8/family1 is initiating major compaction (all files) 2023-03-08 07:55:30,245 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:30,245 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:30,246 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of a50cf6d148fcdc856ce86b2cdca8ceb8/family1 in testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:30,246 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/6379d180c1c74c4ab36bc3e0f489e814] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp, totalSize=26.7 K 2023-03-08 07:55:30,246 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 8986c97ff59f4b56bf600bf2a1520ee7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116957 2023-03-08 07:55:30,247 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting deb2ab70953849c584ea0ec88bf3ed81, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:30,247 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting f8a89224d8404fd3b8cec9768b68df23, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120561 2023-03-08 07:55:30,247 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 57716fd64aec4b8a9f14ced6989e1725, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121978 2023-03-08 07:55:30,248 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 6379d180c1c74c4ab36bc3e0f489e814, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 07:55:30,258 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): a50cf6d148fcdc856ce86b2cdca8ceb8#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:30,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/930c31d1eb7c4b92a609178381b469ea] to archive 2023-03-08 07:55:30,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:30,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/650c0bf692e9431184c7ef97d0ef75ea 2023-03-08 07:55:30,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/bec62040e49a41f3892006664123aba7 2023-03-08 07:55:30,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/6f57e6ccdff045cd8423070fb969e229 2023-03-08 07:55:30,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/ae1cf04043de49c884f5504f54ee4297 2023-03-08 07:55:30,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/930c31d1eb7c4b92a609178381b469ea to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/930c31d1eb7c4b92a609178381b469ea 2023-03-08 07:55:30,668 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del 2023-03-08 07:55:30,669 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del 2023-03-08 07:55:31,484 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e01b4a74c564bf6ab4ba19aae5adca5 2023-03-08 07:55:31,484 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/1e01b4a74c564bf6ab4ba19aae5adca5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/1e01b4a74c564bf6ab4ba19aae5adca5 2023-03-08 07:55:31,488 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e01b4a74c564bf6ab4ba19aae5adca5 2023-03-08 07:55:31,488 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a50cf6d148fcdc856ce86b2cdca8ceb8/family1 of a50cf6d148fcdc856ce86b2cdca8ceb8 into 1e01b4a74c564bf6ab4ba19aae5adca5(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:31,489 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:31,489 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:31,489 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): a50cf6d148fcdc856ce86b2cdca8ceb8/family2 is initiating major compaction (all files) 2023-03-08 07:55:31,489 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:31,489 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:31,489 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of a50cf6d148fcdc856ce86b2cdca8ceb8/family2 in testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:31,489 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/e959ef0924604bb58c6c998337cdd24f] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp, totalSize=25.4 K 2023-03-08 07:55:31,490 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting f46ce0a8709142a28bb3626c172a4a46, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116957 2023-03-08 07:55:31,490 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 7ff98be37e5643c59927680a6da57fdc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:31,490 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting c33d6be82d7e460db1fdbbf7131c6545, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120561 2023-03-08 07:55:31,490 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 4b19bbe0117d46e4937560e8cb137e61, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121978 2023-03-08 07:55:31,491 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting e959ef0924604bb58c6c998337cdd24f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 07:55:31,503 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): a50cf6d148fcdc856ce86b2cdca8ceb8#family2#compaction#110 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:31,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/6379d180c1c74c4ab36bc3e0f489e814] to archive 2023-03-08 07:55:31,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:31,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/8986c97ff59f4b56bf600bf2a1520ee7 2023-03-08 07:55:31,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/deb2ab70953849c584ea0ec88bf3ed81 2023-03-08 07:55:31,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f8a89224d8404fd3b8cec9768b68df23 2023-03-08 07:55:31,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/57716fd64aec4b8a9f14ced6989e1725 2023-03-08 07:55:31,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/6379d180c1c74c4ab36bc3e0f489e814 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/6379d180c1c74c4ab36bc3e0f489e814 2023-03-08 07:55:31,912 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del 2023-03-08 07:55:31,912 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del 2023-03-08 07:55:32,727 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c541977dd38b4e80b02e6c773d534f7d 2023-03-08 07:55:32,727 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/c541977dd38b4e80b02e6c773d534f7d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c541977dd38b4e80b02e6c773d534f7d 2023-03-08 07:55:32,730 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c541977dd38b4e80b02e6c773d534f7d 2023-03-08 07:55:32,731 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a50cf6d148fcdc856ce86b2cdca8ceb8/family2 of a50cf6d148fcdc856ce86b2cdca8ceb8 into c541977dd38b4e80b02e6c773d534f7d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:32,731 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:32,731 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:32,731 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): e9fb34d1dad0ba5071bdda7f618f683e/family1 is initiating major compaction (all files) 2023-03-08 07:55:32,732 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:32,732 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:32,732 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of e9fb34d1dad0ba5071bdda7f618f683e/family1 in testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:32,732 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/b0c84436b1c447429244807dd62c8e4f] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp, totalSize=26.7 K 2023-03-08 07:55:32,732 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 04fda751d9ed4b71bdfbd09e53c14b7b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116959 2023-03-08 07:55:32,732 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 32ba1adf9a024898b96e4d8e4abf926c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:32,733 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting f32fa60784ae4ddabafe1a8293029119, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:32,733 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 97e4fcf5f5f6408f8f0653dcd03bc9dc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:32,733 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b0c84436b1c447429244807dd62c8e4f, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 07:55:32,736 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/e959ef0924604bb58c6c998337cdd24f] to archive 2023-03-08 07:55:32,736 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:32,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/f46ce0a8709142a28bb3626c172a4a46 2023-03-08 07:55:32,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/7ff98be37e5643c59927680a6da57fdc 2023-03-08 07:55:32,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c33d6be82d7e460db1fdbbf7131c6545 2023-03-08 07:55:32,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/4b19bbe0117d46e4937560e8cb137e61 2023-03-08 07:55:32,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/e959ef0924604bb58c6c998337cdd24f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/e959ef0924604bb58c6c998337cdd24f 2023-03-08 07:55:32,745 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): e9fb34d1dad0ba5071bdda7f618f683e#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:33,152 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del 2023-03-08 07:55:33,152 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del 2023-03-08 07:55:33,965 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70db4b776da94cef944786ca68348123 2023-03-08 07:55:33,966 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/70db4b776da94cef944786ca68348123 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/70db4b776da94cef944786ca68348123 2023-03-08 07:55:33,970 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70db4b776da94cef944786ca68348123 2023-03-08 07:55:33,971 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e9fb34d1dad0ba5071bdda7f618f683e/family1 of e9fb34d1dad0ba5071bdda7f618f683e into 70db4b776da94cef944786ca68348123(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:33,971 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:33,971 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:33,971 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): e9fb34d1dad0ba5071bdda7f618f683e/family2 is initiating major compaction (all files) 2023-03-08 07:55:33,971 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:33,971 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:33,971 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of e9fb34d1dad0ba5071bdda7f618f683e/family2 in testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:33,971 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/c26c3c0a78f146899623a949f7744f3f] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp, totalSize=25.4 K 2023-03-08 07:55:33,972 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 9cae59daa977464dbc7c2d8b8a2bed4a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116959 2023-03-08 07:55:33,972 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 7e24d05a9202460e924485dec0ce3271, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:33,972 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b6c0e39cb8fd4b39a6570ccd11d3b508, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:33,973 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 9c31cc68bf324de28df80734eb1c30e2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:33,973 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting c26c3c0a78f146899623a949f7744f3f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 07:55:33,987 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): e9fb34d1dad0ba5071bdda7f618f683e#family2#compaction#112 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:34,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/b0c84436b1c447429244807dd62c8e4f] to archive 2023-03-08 07:55:34,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:34,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/04fda751d9ed4b71bdfbd09e53c14b7b 2023-03-08 07:55:34,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/32ba1adf9a024898b96e4d8e4abf926c 2023-03-08 07:55:34,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/f32fa60784ae4ddabafe1a8293029119 2023-03-08 07:55:34,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/97e4fcf5f5f6408f8f0653dcd03bc9dc 2023-03-08 07:55:34,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/b0c84436b1c447429244807dd62c8e4f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/b0c84436b1c447429244807dd62c8e4f 2023-03-08 07:55:34,397 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del 2023-03-08 07:55:34,397 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del 2023-03-08 07:55:35,210 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 451011b21c93465daea3521b15397448 2023-03-08 07:55:35,211 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/451011b21c93465daea3521b15397448 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/451011b21c93465daea3521b15397448 2023-03-08 07:55:35,215 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 451011b21c93465daea3521b15397448 2023-03-08 07:55:35,216 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e9fb34d1dad0ba5071bdda7f618f683e/family2 of e9fb34d1dad0ba5071bdda7f618f683e into 451011b21c93465daea3521b15397448(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:35,216 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:35,221 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:35,224 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:35,229 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:35,230 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:35,236 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 07:55:35,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/c26c3c0a78f146899623a949f7744f3f] to archive 2023-03-08 07:55:35,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9cae59daa977464dbc7c2d8b8a2bed4a 2023-03-08 07:55:35,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/7e24d05a9202460e924485dec0ce3271 2023-03-08 07:55:35,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/b6c0e39cb8fd4b39a6570ccd11d3b508 2023-03-08 07:55:35,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/9c31cc68bf324de28df80734eb1c30e2 2023-03-08 07:55:35,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/c26c3c0a78f146899623a949f7744f3f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/c26c3c0a78f146899623a949f7744f3f 2023-03-08 07:55:35,271 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 07:55:35,274 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del 2023-03-08 07:55:35,276 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del 2023-03-08 07:55:35,279 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del 2023-03-08 07:55:35,279 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,279 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(139): Connect 0x33019f08 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:35,284 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24074208, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:35,284 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,284 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,284 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,748 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308231ce706b2a140cd8fb9576b01580951 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308231ce706b2a140cd8fb9576b01580951 2023-03-08 07:55:35,749 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,750 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030819d8cfcbbd6b4d9aa7a7505db7e17582 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030819d8cfcbbd6b4d9aa7a7505db7e17582 2023-03-08 07:55:35,750 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,764 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b2804dd8fc8c4893aba1e06ca405e3a4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b2804dd8fc8c4893aba1e06ca405e3a4 2023-03-08 07:55:35,765 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,770 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:35,774 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51996, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:35,780 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:55:35,786 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60662, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:55:35,808 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/60f04331314b43639fa20b0634678584 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 07:55:35,809 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9cfeff1aef8c4262bab32b851decca8b first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 07:55:35,809 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/60f04331314b43639fa20b0634678584}] 2023-03-08 07:55:35,809 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/83b27a005dae42b7977fc7bd2b5b90eb first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 07:55:35,811 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9cfeff1aef8c4262bab32b851decca8b}] 2023-03-08 07:55:35,811 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/83b27a005dae42b7977fc7bd2b5b90eb}] 2023-03-08 07:55:35,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9cfeff1aef8c4262bab32b851decca8b for inclusion in a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:35,816 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/60f04331314b43639fa20b0634678584 for inclusion in e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:35,816 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/83b27a005dae42b7977fc7bd2b5b90eb for inclusion in 2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:35,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 07:55:35,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:35,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/60f04331314b43639fa20b0634678584 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__3ai5u3vv98g4t43jutv0iov8sgppfcg61lgc6mtu1lstapd4u3nttft7gle5tr1/family1/60f04331314b43639fa20b0634678584 2023-03-08 07:55:35,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9cfeff1aef8c4262bab32b851decca8b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__8vrm7c06s9ouot2qdnjpvuu611t6iv05jein7mu5pi0dl2om5fqhl6tkq4gsli23/family1/9cfeff1aef8c4262bab32b851decca8b 2023-03-08 07:55:35,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/83b27a005dae42b7977fc7bd2b5b90eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__ap14egpii3tefp4tmsgfilqbl3dm6hgerc8c5ua8nj8r21j0kn8sjs83u665mrg2/family1/83b27a005dae42b7977fc7bd2b5b90eb 2023-03-08 07:55:35,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__ap14egpii3tefp4tmsgfilqbl3dm6hgerc8c5ua8nj8r21j0kn8sjs83u665mrg2/family1/83b27a005dae42b7977fc7bd2b5b90eb as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_ 2023-03-08 07:55:35,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__3ai5u3vv98g4t43jutv0iov8sgppfcg61lgc6mtu1lstapd4u3nttft7gle5tr1/family1/60f04331314b43639fa20b0634678584 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_ 2023-03-08 07:55:35,824 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__ap14egpii3tefp4tmsgfilqbl3dm6hgerc8c5ua8nj8r21j0kn8sjs83u665mrg2/family1/83b27a005dae42b7977fc7bd2b5b90eb into 2fb06f0faeeffea363aea80614ef1a51/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_ - updating store file list. 2023-03-08 07:55:35,824 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__3ai5u3vv98g4t43jutv0iov8sgppfcg61lgc6mtu1lstapd4u3nttft7gle5tr1/family1/60f04331314b43639fa20b0634678584 into e9fb34d1dad0ba5071bdda7f618f683e/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_ - updating store file list. 2023-03-08 07:55:35,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__8vrm7c06s9ouot2qdnjpvuu611t6iv05jein7mu5pi0dl2om5fqhl6tkq4gsli23/family1/9cfeff1aef8c4262bab32b851decca8b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_ 2023-03-08 07:55:35,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__8vrm7c06s9ouot2qdnjpvuu611t6iv05jein7mu5pi0dl2om5fqhl6tkq4gsli23/family1/9cfeff1aef8c4262bab32b851decca8b into a50cf6d148fcdc856ce86b2cdca8ceb8/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_ - updating store file list. 2023-03-08 07:55:35,828 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_ into e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:35,828 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__3ai5u3vv98g4t43jutv0iov8sgppfcg61lgc6mtu1lstapd4u3nttft7gle5tr1/family1/60f04331314b43639fa20b0634678584 into e9fb34d1dad0ba5071bdda7f618f683e/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_) 2023-03-08 07:55:35,828 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_ into 2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:35,828 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__ap14egpii3tefp4tmsgfilqbl3dm6hgerc8c5ua8nj8r21j0kn8sjs83u665mrg2/family1/83b27a005dae42b7977fc7bd2b5b90eb into 2fb06f0faeeffea363aea80614ef1a51/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_) 2023-03-08 07:55:35,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__ap14egpii3tefp4tmsgfilqbl3dm6hgerc8c5ua8nj8r21j0kn8sjs83u665mrg2/family1/83b27a005dae42b7977fc7bd2b5b90eb 2023-03-08 07:55:35,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__3ai5u3vv98g4t43jutv0iov8sgppfcg61lgc6mtu1lstapd4u3nttft7gle5tr1/family1/60f04331314b43639fa20b0634678584 2023-03-08 07:55:35,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_ into a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:35,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__8vrm7c06s9ouot2qdnjpvuu611t6iv05jein7mu5pi0dl2om5fqhl6tkq4gsli23/family1/9cfeff1aef8c4262bab32b851decca8b into a50cf6d148fcdc856ce86b2cdca8ceb8/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_) 2023-03-08 07:55:35,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__8vrm7c06s9ouot2qdnjpvuu611t6iv05jein7mu5pi0dl2om5fqhl6tkq4gsli23/family1/9cfeff1aef8c4262bab32b851decca8b 2023-03-08 07:55:35,833 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,833 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,833 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,834 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,834 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,834 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,836 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 2023-03-08 07:55:35,836 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b 2023-03-08 07:55:35,836 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 2023-03-08 07:55:35,837 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 2023-03-08 07:55:35,838 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a 2023-03-08 07:55:35,838 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb 2023-03-08 07:55:35,838 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 2023-03-08 07:55:35,839 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 2023-03-08 07:55:35,839 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 2023-03-08 07:55:35,840 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 2023-03-08 07:55:35,840 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 07:55:35,840 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 07:55:35,841 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe 2023-03-08 07:55:35,841 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 07:55:35,841 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 07:55:35,841 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b 2023-03-08 07:55:35,841 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,841 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 07:55:35,841 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 07:55:35,842 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,842 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,842 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:35,842 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:35,842 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del] 2023-03-08 07:55:35,843 DEBUG [Listener at localhost/42999] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,844 DEBUG [Listener at localhost/42999] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308640a19b8d81346998212e35c4fc59d6a_del 2023-03-08 07:55:35,844 INFO [Listener at localhost/42999] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 07:55:35,844 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del] 2023-03-08 07:55:35,845 DEBUG [Listener at localhost/42999] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,846 DEBUG [Listener at localhost/42999] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308fe2b18a547c94e9d86900c283935f3a3_del 2023-03-08 07:55:35,846 INFO [Listener at localhost/42999] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 07:55:35,846 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del] 2023-03-08 07:55:35,846 DEBUG [Listener at localhost/42999] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:35,848 DEBUG [Listener at localhost/42999] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030834028e7d119646d59ccc50a4db374495_del 2023-03-08 07:55:35,848 INFO [Listener at localhost/42999] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 07:55:35,851 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:35,853 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:35,857 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:35,858 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:35,863 INFO [Listener at localhost/42999] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-08 07:55:35,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-08 07:55:35,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-08 07:55:35,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 07:55:35,867 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262135867"}]},"ts":"1678262135867"} 2023-03-08 07:55:35,868 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-08 07:55:35,871 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-08 07:55:35,871 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, UNASSIGN}] 2023-03-08 07:55:35,873 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, UNASSIGN 2023-03-08 07:55:35,873 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, UNASSIGN 2023-03-08 07:55:35,873 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, UNASSIGN 2023-03-08 07:55:35,873 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=a50cf6d148fcdc856ce86b2cdca8ceb8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:35,873 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=e9fb34d1dad0ba5071bdda7f618f683e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:35,873 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=2fb06f0faeeffea363aea80614ef1a51, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:35,873 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262135873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262135873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262135873"}]},"ts":"1678262135873"} 2023-03-08 07:55:35,873 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262135873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262135873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262135873"}]},"ts":"1678262135873"} 2023-03-08 07:55:35,873 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262135873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262135873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262135873"}]},"ts":"1678262135873"} 2023-03-08 07:55:35,874 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:35,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:35,875 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:35,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 07:55:35,997 ERROR [ProcedureDispatcherTimeoutThread] procedure2.RemoteProcedureDispatcher$TimeoutExecutorThread(317): DelayQueue for RemoteProcedureDispatcher is not empty when timed waiting elapsed. If this is repeated consistently, it means no element is getting expired from the queue and it might freeze the system. Queue: [containedObject=jenkins-hbase4.apache.org,33669,1678262016084, timeout=1678262136026, delay=29, operations=[pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084, pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084, pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084]] 2023-03-08 07:55:36,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:36,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e9fb34d1dad0ba5071bdda7f618f683e, disabling compactions & flushes 2023-03-08 07:55:36,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:36,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:36,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. after waiting 0 ms 2023-03-08 07:55:36,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:36,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 07:55:36,032 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:36,032 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:36,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:36,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:36,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a50cf6d148fcdc856ce86b2cdca8ceb8, disabling compactions & flushes 2023-03-08 07:55:36,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:36,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:36,034 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=e9fb34d1dad0ba5071bdda7f618f683e, regionState=CLOSED 2023-03-08 07:55:36,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. after waiting 0 ms 2023-03-08 07:55:36,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:36,034 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262136034"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262136034"}]},"ts":"1678262136034"} 2023-03-08 07:55:36,037 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-03-08 07:55:36,037 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084 in 161 msec 2023-03-08 07:55:36,038 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, UNASSIGN in 166 msec 2023-03-08 07:55:36,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 07:55:36,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:36,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:36,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:36,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:36,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2fb06f0faeeffea363aea80614ef1a51, disabling compactions & flushes 2023-03-08 07:55:36,041 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=a50cf6d148fcdc856ce86b2cdca8ceb8, regionState=CLOSED 2023-03-08 07:55:36,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:36,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:36,041 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262136041"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262136041"}]},"ts":"1678262136041"} 2023-03-08 07:55:36,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. after waiting 0 ms 2023-03-08 07:55:36,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:36,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-03-08 07:55:36,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 168 msec 2023-03-08 07:55:36,044 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, UNASSIGN in 172 msec 2023-03-08 07:55:36,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 07:55:36,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:36,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:36,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:36,048 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=2fb06f0faeeffea363aea80614ef1a51, regionState=CLOSED 2023-03-08 07:55:36,048 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262136048"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262136048"}]},"ts":"1678262136048"} 2023-03-08 07:55:36,050 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-03-08 07:55:36,050 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084 in 174 msec 2023-03-08 07:55:36,051 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-03-08 07:55:36,051 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, UNASSIGN in 179 msec 2023-03-08 07:55:36,052 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262136052"}]},"ts":"1678262136052"} 2023-03-08 07:55:36,053 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-08 07:55:36,054 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-08 07:55:36,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 191 msec 2023-03-08 07:55:36,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 07:55:36,169 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-08 07:55:36,179 INFO [Listener at localhost/42999] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:36,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:36,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678262136180 (current time:1678262136180). 2023-03-08 07:55:36,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 07:55:36,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] zookeeper.ReadOnlyZKClient(139): Connect 0x78b6ffa7 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:36,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74f399d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:36,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:36,188 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52000, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:36,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x78b6ffa7 to 127.0.0.1:52387 2023-03-08 07:55:36,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:36,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 07:55:36,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:36,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 07:55:36,191 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 07:55:36,192 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678262736192 2023-03-08 07:55:36,192 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678262123778-1678262136179 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 07:55:36,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:36,192 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:36,193 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 07:55:36,293 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:36,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:36,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:36,294 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 07:55:36,494 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:36,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:36,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:36,496 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 07:55:36,599 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 07:55:36,601 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/739644265f1e4f6b899269a64eed8f18, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_] hfiles 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/739644265f1e4f6b899269a64eed8f18 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/1e01b4a74c564bf6ab4ba19aae5adca5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_] hfiles 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/70db4b776da94cef944786ca68348123, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_] hfiles 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/1e01b4a74c564bf6ab4ba19aae5adca5 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_ 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_ 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308231ce706b2a140cd8fb9576b01580951, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030819d8cfcbbd6b4d9aa7a7505db7e17582, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b2804dd8fc8c4893aba1e06ca405e3a4] hfiles 2023-03-08 07:55:36,604 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/70db4b776da94cef944786ca68348123 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308231ce706b2a140cd8fb9576b01580951 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_ 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030819d8cfcbbd6b4d9aa7a7505db7e17582 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b2804dd8fc8c4893aba1e06ca405e3a4 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c541977dd38b4e80b02e6c773d534f7d] hfiles 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/3178b0499df84c24b7261c416aa4b750] hfiles 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c541977dd38b4e80b02e6c773d534f7d 2023-03-08 07:55:36,605 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/3178b0499df84c24b7261c416aa4b750 2023-03-08 07:55:36,606 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/451011b21c93465daea3521b15397448] hfiles 2023-03-08 07:55:36,606 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/451011b21c93465daea3521b15397448 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303088972274e07814bcb8afc2d88bbee21e2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308bc16c75ca33c4bf7bae11cc42bc1c2a8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d07ea5cdca74445f824b90fa36aca6ef, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd902299661f4b9abddc12dccc114e5b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030897dbef609cad42e29e296927c255dd6f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b901881636af44f69d119f6558b2ab98, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308db8d12fe113142e58b12a3aa4db49c42, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308fcbe739713b745cab5958eadbc0dbd42, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303086439fd156cdc46e484e3bdd40ca1b107, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030894dc8b9e31a3439c9805e0e258f462e7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b000d47aaa2d4ff4a3800fc3e2b0c824, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d9ea1e6fd03e4340b7e67abe1a346544] hfiles 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303088972274e07814bcb8afc2d88bbee21e2 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308bc16c75ca33c4bf7bae11cc42bc1c2a8 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d07ea5cdca74445f824b90fa36aca6ef 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd902299661f4b9abddc12dccc114e5b 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030897dbef609cad42e29e296927c255dd6f 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b901881636af44f69d119f6558b2ab98 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308db8d12fe113142e58b12a3aa4db49c42 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308fcbe739713b745cab5958eadbc0dbd42 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303086439fd156cdc46e484e3bdd40ca1b107 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030894dc8b9e31a3439c9805e0e258f462e7 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b000d47aaa2d4ff4a3800fc3e2b0c824 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del 2023-03-08 07:55:36,611 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d9ea1e6fd03e4340b7e67abe1a346544 2023-03-08 07:55:36,796 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:36,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:36,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:36,797 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 07:55:37,014 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-08 07:55:37,014 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:37,015 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:37,297 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:37,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:37,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 07:55:37,298 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 07:55:37,447 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678262123778-1678262136179 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:38,262 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678262123778-1678262136179 of table testCompactionWithHFileLink completed 2023-03-08 07:55:38,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:38,262 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:38,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678262123778-1678262136179 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678262136192Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678262136599 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678262123778-1678262136179 at 1678262137014 (+415 ms)Loading Region manifests for hbase-failsafe-snaptb-1678262123778-1678262136179 at 1678262137014Writing data manifest for hbase-failsafe-snaptb-1678262123778-1678262136179 at 1678262137017 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1678262123778-1678262136179 at 1678262137423 (+406 ms)Snapshot hbase-failsafe-snaptb-1678262123778-1678262136179 of table testCompactionWithHFileLink completed at 1678262138262 (+839 ms) 2023-03-08 07:55:38,264 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 07:55:38,265 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0740 sec 2023-03-08 07:55:38,299 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 07:55:38,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 07:55:38,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678262123778-1678262136179 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 07:55:38,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678262123778" table: "testCompactionWithHFileLink" creation_time: 1678262123788 type: FLUSH version: 2 ttl: 0 ) 2023-03-08 07:55:38,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678262123778 as table=testCompactionWithHFileLink 2023-03-08 07:55:38,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 07:55:38,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 07:55:38,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 07:55:38,762 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 07:55:38,762 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 07:55:38,763 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 07:55:38,763 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 07:55:38,768 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678262123778" table: "testCompactionWithHFileLink" creation_time: 1678262123788 type: FLUSH version: 2 ttl: 0 2023-03-08 07:55:38,769 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink 2023-03-08 07:55:38,773 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-08 07:55:38,777 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030819d8cfcbbd6b4d9aa7a7505db7e17582 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030819d8cfcbbd6b4d9aa7a7505db7e17582 2023-03-08 07:55:38,778 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b2804dd8fc8c4893aba1e06ca405e3a4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b2804dd8fc8c4893aba1e06ca405e3a4 2023-03-08 07:55:38,779 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308231ce706b2a140cd8fb9576b01580951 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308231ce706b2a140cd8fb9576b01580951 2023-03-08 07:55:38,779 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,783 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,786 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,789 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,792 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,795 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,798 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,801 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,804 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,806 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,809 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,812 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b of snapshot=snaptb-1678262123778 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 07:55:38,816 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308c4c82d38b5b643569b5bab25e28f447b_del 2023-03-08 07:55:38,817 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080584c39c42c74e6fa490801372e4f991_del 2023-03-08 07:55:38,818 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303080235cc3e3d2f4848849ba8592253baf0_del 2023-03-08 07:55:38,818 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:38,818 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:38,818 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:38,821 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/d7b526e89939489ba8bb8a6c7d96a0ba_SeqId_29_ 2023-03-08 07:55:38,822 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/1e01b4a74c564bf6ab4ba19aae5adca5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/1e01b4a74c564bf6ab4ba19aae5adca5 2023-03-08 07:55:38,822 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/70db4b776da94cef944786ca68348123 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/70db4b776da94cef944786ca68348123 2023-03-08 07:55:38,822 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/739644265f1e4f6b899269a64eed8f18 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/739644265f1e4f6b899269a64eed8f18 2023-03-08 07:55:38,823 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b1a0e74efa4c4846bc9024a6de7e8074 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,823 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/85ccb8e4667643468b2dc3ab5ba75d41_SeqId_29_ 2023-03-08 07:55:38,823 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 04fda751d9ed4b71bdfbd09e53c14b7b of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,823 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6fa1cf4db8646d08f55042e3bb37d0b_SeqId_29_ 2023-03-08 07:55:38,823 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8986c97ff59f4b56bf600bf2a1520ee7 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,826 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4d800d09acb843c68c40907b9749c4b0 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,826 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 32ba1adf9a024898b96e4d8e4abf926c of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,827 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink deb2ab70953849c584ea0ec88bf3ed81 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,830 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5e9ed625067b4b08a3f7b952e348a0b7 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,830 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f32fa60784ae4ddabafe1a8293029119 of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,830 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f8a89224d8404fd3b8cec9768b68df23 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,833 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 97e4fcf5f5f6408f8f0653dcd03bc9dc of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,833 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c15382d6134c492ab93f119690918ab1 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,836 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 57716fd64aec4b8a9f14ced6989e1725 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,845 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/451011b21c93465daea3521b15397448 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/451011b21c93465daea3521b15397448 2023-03-08 07:55:38,845 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9cae59daa977464dbc7c2d8b8a2bed4a of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,846 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/3178b0499df84c24b7261c416aa4b750 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/3178b0499df84c24b7261c416aa4b750 2023-03-08 07:55:38,846 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 650c0bf692e9431184c7ef97d0ef75ea of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,846 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c541977dd38b4e80b02e6c773d534f7d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/c541977dd38b4e80b02e6c773d534f7d 2023-03-08 07:55:38,846 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f46ce0a8709142a28bb3626c172a4a46 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,849 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7e24d05a9202460e924485dec0ce3271 of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,849 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bec62040e49a41f3892006664123aba7 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,849 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7ff98be37e5643c59927680a6da57fdc of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,852 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b6c0e39cb8fd4b39a6570ccd11d3b508 of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,853 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6f57e6ccdff045cd8423070fb969e229 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,853 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c33d6be82d7e460db1fdbbf7131c6545 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,856 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ae1cf04043de49c884f5504f54ee4297 of snapshot=snaptb-1678262123778 to region=2fb06f0faeeffea363aea80614ef1a51 table=testCompactionWithHFileLink 2023-03-08 07:55:38,857 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9c31cc68bf324de28df80734eb1c30e2 of snapshot=snaptb-1678262123778 to region=e9fb34d1dad0ba5071bdda7f618f683e table=testCompactionWithHFileLink 2023-03-08 07:55:38,857 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4b19bbe0117d46e4937560e8cb137e61 of snapshot=snaptb-1678262123778 to region=a50cf6d148fcdc856ce86b2cdca8ceb8 table=testCompactionWithHFileLink 2023-03-08 07:55:38,860 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678262123778" table: "testCompactionWithHFileLink" creation_time: 1678262123788 type: FLUSH version: 2 ttl: 0 2023-03-08 07:55:38,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678262138861"}]},"ts":"9223372036854775807"} 2023-03-08 07:55:38,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678262138861"}]},"ts":"9223372036854775807"} 2023-03-08 07:55:38,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678262138861"}]},"ts":"9223372036854775807"} 2023-03-08 07:55:38,863 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-08 07:55:38,863 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2fb06f0faeeffea363aea80614ef1a51, NAME => 'testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => a50cf6d148fcdc856ce86b2cdca8ceb8, NAME => 'testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => e9fb34d1dad0ba5071bdda7f618f683e, NAME => 'testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.', STARTKEY => '2', ENDKEY => ''}] 2023-03-08 07:55:38,864 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262138862"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262138862"}]},"ts":"1678262138862"} 2023-03-08 07:55:38,864 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262138862"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262138862"}]},"ts":"1678262138862"} 2023-03-08 07:55:38,864 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262138862"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262138862"}]},"ts":"1678262138862"} 2023-03-08 07:55:38,865 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 07:55:38,865 INFO [PEWorker-4] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-08 07:55:38,865 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 2fb06f0faeeffea363aea80614ef1a51, NAME => 'testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => a50cf6d148fcdc856ce86b2cdca8ceb8, NAME => 'testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => e9fb34d1dad0ba5071bdda7f618f683e, NAME => 'testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.', STARTKEY => '2', ENDKEY => ''}] 2023-03-08 07:55:38,866 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678262123778 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-08 07:55:38,867 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678262123778" table: "testCompactionWithHFileLink" creation_time: 1678262123788 type: FLUSH version: 2 ttl: 0 ) in 528 msec 2023-03-08 07:55:38,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 07:55:38,957 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-08 07:55:38,957 INFO [Listener at localhost/42999] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:38,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678262123778-1678262136179" 2023-03-08 07:55:38,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678262123778-1678262136179 2023-03-08 07:55:38,968 INFO [Listener at localhost/42999] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-08 07:55:38,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-08 07:55:38,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-08 07:55:38,985 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-08 07:55:38,985 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262138985"}]},"ts":"1678262138985"} 2023-03-08 07:55:38,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 07:55:38,986 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 07:55:38,989 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-08 07:55:38,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN}] 2023-03-08 07:55:38,990 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN 2023-03-08 07:55:38,991 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN 2023-03-08 07:55:38,991 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN 2023-03-08 07:55:38,991 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 07:55:38,991 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 07:55:38,991 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 07:55:39,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 07:55:39,143 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a50cf6d148fcdc856ce86b2cdca8ceb8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:39,143 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=e9fb34d1dad0ba5071bdda7f618f683e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:39,143 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262139143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262139143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262139143"}]},"ts":"1678262139143"} 2023-03-08 07:55:39,143 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=2fb06f0faeeffea363aea80614ef1a51, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:39,143 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262139143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262139143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262139143"}]},"ts":"1678262139143"} 2023-03-08 07:55:39,143 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262139143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262139143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262139143"}]},"ts":"1678262139143"} 2023-03-08 07:55:39,144 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=98, state=RUNNABLE; OpenRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:39,145 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:39,145 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:39,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 07:55:39,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:39,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2fb06f0faeeffea363aea80614ef1a51, NAME => 'testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 07:55:39,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:39,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,300 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,301 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:39,301 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:39,301 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2fb06f0faeeffea363aea80614ef1a51 columnFamilyName family1 2023-03-08 07:55:39,316 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-4d800d09acb843c68c40907b9749c4b0 2023-03-08 07:55:39,321 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-5e9ed625067b4b08a3f7b952e348a0b7 2023-03-08 07:55:39,326 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-b1a0e74efa4c4846bc9024a6de7e8074 2023-03-08 07:55:39,331 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-c15382d6134c492ab93f119690918ab1 2023-03-08 07:55:39,331 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(310): Store=2fb06f0faeeffea363aea80614ef1a51/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:39,332 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,332 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2 2023-03-08 07:55:39,333 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2 2023-03-08 07:55:39,333 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2fb06f0faeeffea363aea80614ef1a51 columnFamilyName family2 2023-03-08 07:55:39,339 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-650c0bf692e9431184c7ef97d0ef75ea 2023-03-08 07:55:39,344 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-6f57e6ccdff045cd8423070fb969e229 2023-03-08 07:55:39,349 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-ae1cf04043de49c884f5504f54ee4297 2023-03-08 07:55:39,353 DEBUG [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-bec62040e49a41f3892006664123aba7 2023-03-08 07:55:39,354 INFO [StoreOpener-2fb06f0faeeffea363aea80614ef1a51-1] regionserver.HStore(310): Store=2fb06f0faeeffea363aea80614ef1a51/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:39,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:39,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:55:39,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2fb06f0faeeffea363aea80614ef1a51; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69563450, jitterRate=0.03657618165016174}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:39,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:39,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51., pid=103, masterSystemTime=1678262139296 2023-03-08 07:55:39,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:39,359 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 07:55:39,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 07:55:39,359 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 07:55:39,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:39,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:39,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:39,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e9fb34d1dad0ba5071bdda7f618f683e, NAME => 'testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 07:55:39,363 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=2fb06f0faeeffea363aea80614ef1a51, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:39,363 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262139362"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262139362"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262139362"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262139362"}]},"ts":"1678262139362"} 2023-03-08 07:55:39,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:39,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,363 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 07:55:39,363 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1912): 2fb06f0faeeffea363aea80614ef1a51/family1 is initiating minor compaction (all files) 2023-03-08 07:55:39,364 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2fb06f0faeeffea363aea80614ef1a51/family1 in testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:39,364 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-b1a0e74efa4c4846bc9024a6de7e8074, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-4d800d09acb843c68c40907b9749c4b0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-5e9ed625067b4b08a3f7b952e348a0b7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-c15382d6134c492ab93f119690918ab1] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp, totalSize=21.6 K 2023-03-08 07:55:39,364 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 07:55:39,364 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1912): 2fb06f0faeeffea363aea80614ef1a51/family2 is initiating minor compaction (all files) 2023-03-08 07:55:39,364 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2fb06f0faeeffea363aea80614ef1a51/family2 in testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:55:39,364 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-650c0bf692e9431184c7ef97d0ef75ea, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-bec62040e49a41f3892006664123aba7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-6f57e6ccdff045cd8423070fb969e229, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-ae1cf04043de49c884f5504f54ee4297] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp, totalSize=20.4 K 2023-03-08 07:55:39,365 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,365 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-b1a0e74efa4c4846bc9024a6de7e8074, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116958 2023-03-08 07:55:39,365 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-650c0bf692e9431184c7ef97d0ef75ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116958 2023-03-08 07:55:39,366 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-08 07:55:39,366 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-4d800d09acb843c68c40907b9749c4b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:39,366 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 2fb06f0faeeffea363aea80614ef1a51, server=jenkins-hbase4.apache.org,33669,1678262016084 in 219 msec 2023-03-08 07:55:39,366 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:39,366 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:39,367 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-5e9ed625067b4b08a3f7b952e348a0b7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:39,367 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9fb34d1dad0ba5071bdda7f618f683e columnFamilyName family1 2023-03-08 07:55:39,367 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2fb06f0faeeffea363aea80614ef1a51, ASSIGN in 377 msec 2023-03-08 07:55:39,367 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-c15382d6134c492ab93f119690918ab1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:39,369 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-bec62040e49a41f3892006664123aba7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:39,370 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-6f57e6ccdff045cd8423070fb969e229, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:39,371 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-ae1cf04043de49c884f5504f54ee4297, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:39,382 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-04fda751d9ed4b71bdfbd09e53c14b7b 2023-03-08 07:55:39,392 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-32ba1adf9a024898b96e4d8e4abf926c 2023-03-08 07:55:39,406 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-97e4fcf5f5f6408f8f0653dcd03bc9dc 2023-03-08 07:55:39,414 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2fb06f0faeeffea363aea80614ef1a51#family1#compaction#113 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:39,423 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-f32fa60784ae4ddabafe1a8293029119 2023-03-08 07:55:39,423 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(310): Store=e9fb34d1dad0ba5071bdda7f618f683e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:39,423 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,425 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2 2023-03-08 07:55:39,425 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2 2023-03-08 07:55:39,425 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] throttle.PressureAwareThroughputController(145): 2fb06f0faeeffea363aea80614ef1a51#family2#compaction#114 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:39,425 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9fb34d1dad0ba5071bdda7f618f683e columnFamilyName family2 2023-03-08 07:55:39,433 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-7e24d05a9202460e924485dec0ce3271 2023-03-08 07:55:39,437 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9c31cc68bf324de28df80734eb1c30e2 2023-03-08 07:55:39,441 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9cae59daa977464dbc7c2d8b8a2bed4a 2023-03-08 07:55:39,445 DEBUG [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-b6c0e39cb8fd4b39a6570ccd11d3b508 2023-03-08 07:55:39,445 INFO [StoreOpener-e9fb34d1dad0ba5071bdda7f618f683e-1] regionserver.HStore(310): Store=e9fb34d1dad0ba5071bdda7f618f683e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:39,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:39,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:55:39,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e9fb34d1dad0ba5071bdda7f618f683e; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74942977, jitterRate=0.11673738062381744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:39,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:39,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e., pid=102, masterSystemTime=1678262139296 2023-03-08 07:55:39,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 07:55:39,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 07:55:39,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:39,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:39,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:39,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a50cf6d148fcdc856ce86b2cdca8ceb8, NAME => 'testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 07:55:39,453 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=e9fb34d1dad0ba5071bdda7f618f683e, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:39,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:39,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262139453"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262139453"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262139453"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262139453"}]},"ts":"1678262139453"} 2023-03-08 07:55:39,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,454 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,455 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:39,455 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:39,456 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a50cf6d148fcdc856ce86b2cdca8ceb8 columnFamilyName family1 2023-03-08 07:55:39,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-03-08 07:55:39,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure e9fb34d1dad0ba5071bdda7f618f683e, server=jenkins-hbase4.apache.org,33669,1678262016084 in 309 msec 2023-03-08 07:55:39,456 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e9fb34d1dad0ba5071bdda7f618f683e, ASSIGN in 467 msec 2023-03-08 07:55:39,461 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-57716fd64aec4b8a9f14ced6989e1725 2023-03-08 07:55:39,467 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-8986c97ff59f4b56bf600bf2a1520ee7 2023-03-08 07:55:39,475 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-deb2ab70953849c584ea0ec88bf3ed81 2023-03-08 07:55:39,481 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f8a89224d8404fd3b8cec9768b68df23 2023-03-08 07:55:39,481 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(310): Store=a50cf6d148fcdc856ce86b2cdca8ceb8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:39,481 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,482 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2 2023-03-08 07:55:39,482 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2 2023-03-08 07:55:39,482 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a50cf6d148fcdc856ce86b2cdca8ceb8 columnFamilyName family2 2023-03-08 07:55:39,489 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-4b19bbe0117d46e4937560e8cb137e61 2023-03-08 07:55:39,496 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-7ff98be37e5643c59927680a6da57fdc 2023-03-08 07:55:39,503 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-c33d6be82d7e460db1fdbbf7131c6545 2023-03-08 07:55:39,510 DEBUG [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(539): loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f46ce0a8709142a28bb3626c172a4a46 2023-03-08 07:55:39,510 INFO [StoreOpener-a50cf6d148fcdc856ce86b2cdca8ceb8-1] regionserver.HStore(310): Store=a50cf6d148fcdc856ce86b2cdca8ceb8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:39,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:39,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:55:39,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a50cf6d148fcdc856ce86b2cdca8ceb8; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71127480, jitterRate=0.05988204479217529}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:39,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:39,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8., pid=101, masterSystemTime=1678262139296 2023-03-08 07:55:39,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 07:55:39,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 07:55:39,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:39,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:39,517 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a50cf6d148fcdc856ce86b2cdca8ceb8, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:39,518 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678262139517"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262139517"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262139517"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262139517"}]},"ts":"1678262139517"} 2023-03-08 07:55:39,520 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=98 2023-03-08 07:55:39,520 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=98, state=SUCCESS; OpenRegionProcedure a50cf6d148fcdc856ce86b2cdca8ceb8, server=jenkins-hbase4.apache.org,33669,1678262016084 in 375 msec 2023-03-08 07:55:39,521 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=98, resume processing ppid=97 2023-03-08 07:55:39,521 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a50cf6d148fcdc856ce86b2cdca8ceb8, ASSIGN in 531 msec 2023-03-08 07:55:39,521 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262139521"}]},"ts":"1678262139521"} 2023-03-08 07:55:39,522 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 07:55:39,524 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-08 07:55:39,525 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 548 msec 2023-03-08 07:55:39,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 07:55:39,589 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-08 07:55:39,602 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:39,610 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:39,621 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:39,622 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:39,628 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 07:55:39,641 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 07:55:39,641 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:39,641 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(139): Connect 0x680094d2 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:39,645 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17383e33, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:39,645 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:39,645 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:39,645 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:40,533 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308aeee04629ff8429997ab43cd9035192e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308aeee04629ff8429997ab43cd9035192e 2023-03-08 07:55:40,534 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308cd6f3b5ef2514bd782de1fb1c9b4c9e3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308cd6f3b5ef2514bd782de1fb1c9b4c9e3 2023-03-08 07:55:40,534 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,535 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,535 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a740307ab12a413f8dc37f225b9e8730 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308a740307ab12a413f8dc37f225b9e8730 2023-03-08 07:55:40,536 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,548 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:40,551 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52012, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:40,562 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:55:40,570 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60672, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:55:40,594 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/700252c6dee54061a2d7ff3b3850248e first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-08 07:55:40,595 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/700252c6dee54061a2d7ff3b3850248e}] 2023-03-08 07:55:40,601 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/ab9bcbc3a07546e19f2d287c74db321a first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-08 07:55:40,602 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/ab9bcbc3a07546e19f2d287c74db321a}] 2023-03-08 07:55:40,602 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/87bc421d46b34b279e223df3e87c69eb first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-08 07:55:40,605 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=32 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/87bc421d46b34b279e223df3e87c69eb}] 2023-03-08 07:55:40,608 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/700252c6dee54061a2d7ff3b3850248e for inclusion in e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:40,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/ab9bcbc3a07546e19f2d287c74db321a for inclusion in 2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:40,609 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/87bc421d46b34b279e223df3e87c69eb for inclusion in a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:40,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-08 07:55:40,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-08 07:55:40,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 07:55:40,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 07:55:40,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-08 07:55:40,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:40,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:40,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 07:55:40,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:40,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/700252c6dee54061a2d7ff3b3850248e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__jl75f5lueqe0ip7t5o9q8gvtod5ikt0j701ifavpj4uk50qncfl9a3v3689g83u9/family1/700252c6dee54061a2d7ff3b3850248e 2023-03-08 07:55:40,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/ab9bcbc3a07546e19f2d287c74db321a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__5k8uh9q6945icvrau9u23u1504p8mg52gjckh65gjts63mq7231hd661pj8fpker/family1/ab9bcbc3a07546e19f2d287c74db321a 2023-03-08 07:55:40,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/87bc421d46b34b279e223df3e87c69eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__smnjsim3n7jd3mc7npar95043fs33pijt7l9r356m0nannhjvsqsm2ndmbp2rpti/family1/87bc421d46b34b279e223df3e87c69eb 2023-03-08 07:55:40,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__jl75f5lueqe0ip7t5o9q8gvtod5ikt0j701ifavpj4uk50qncfl9a3v3689g83u9/family1/700252c6dee54061a2d7ff3b3850248e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_ 2023-03-08 07:55:40,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__5k8uh9q6945icvrau9u23u1504p8mg52gjckh65gjts63mq7231hd661pj8fpker/family1/ab9bcbc3a07546e19f2d287c74db321a as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/0698851c6727496f8b26e9929e9b9ae9_SeqId_34_ 2023-03-08 07:55:40,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__smnjsim3n7jd3mc7npar95043fs33pijt7l9r356m0nannhjvsqsm2ndmbp2rpti/family1/87bc421d46b34b279e223df3e87c69eb as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_ 2023-03-08 07:55:40,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__jl75f5lueqe0ip7t5o9q8gvtod5ikt0j701ifavpj4uk50qncfl9a3v3689g83u9/family1/700252c6dee54061a2d7ff3b3850248e into e9fb34d1dad0ba5071bdda7f618f683e/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_ - updating store file list. 2023-03-08 07:55:40,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__5k8uh9q6945icvrau9u23u1504p8mg52gjckh65gjts63mq7231hd661pj8fpker/family1/ab9bcbc3a07546e19f2d287c74db321a into 2fb06f0faeeffea363aea80614ef1a51/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/0698851c6727496f8b26e9929e9b9ae9_SeqId_34_ - updating store file list. 2023-03-08 07:55:40,618 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__smnjsim3n7jd3mc7npar95043fs33pijt7l9r356m0nannhjvsqsm2ndmbp2rpti/family1/87bc421d46b34b279e223df3e87c69eb into a50cf6d148fcdc856ce86b2cdca8ceb8/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_ - updating store file list. 2023-03-08 07:55:40,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_ into e9fb34d1dad0ba5071bdda7f618f683e/family1 2023-03-08 07:55:40,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__jl75f5lueqe0ip7t5o9q8gvtod5ikt0j701ifavpj4uk50qncfl9a3v3689g83u9/family1/700252c6dee54061a2d7ff3b3850248e into e9fb34d1dad0ba5071bdda7f618f683e/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_) 2023-03-08 07:55:40,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/0698851c6727496f8b26e9929e9b9ae9_SeqId_34_ into 2fb06f0faeeffea363aea80614ef1a51/family1 2023-03-08 07:55:40,622 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_ into a50cf6d148fcdc856ce86b2cdca8ceb8/family1 2023-03-08 07:55:40,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__5k8uh9q6945icvrau9u23u1504p8mg52gjckh65gjts63mq7231hd661pj8fpker/family1/ab9bcbc3a07546e19f2d287c74db321a into 2fb06f0faeeffea363aea80614ef1a51/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/0698851c6727496f8b26e9929e9b9ae9_SeqId_34_) 2023-03-08 07:55:40,622 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__smnjsim3n7jd3mc7npar95043fs33pijt7l9r356m0nannhjvsqsm2ndmbp2rpti/family1/87bc421d46b34b279e223df3e87c69eb into a50cf6d148fcdc856ce86b2cdca8ceb8/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_) 2023-03-08 07:55:40,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__jl75f5lueqe0ip7t5o9q8gvtod5ikt0j701ifavpj4uk50qncfl9a3v3689g83u9/family1/700252c6dee54061a2d7ff3b3850248e 2023-03-08 07:55:40,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:40,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__5k8uh9q6945icvrau9u23u1504p8mg52gjckh65gjts63mq7231hd661pj8fpker/family1/ab9bcbc3a07546e19f2d287c74db321a 2023-03-08 07:55:40,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testCompactionWithHFileLink__smnjsim3n7jd3mc7npar95043fs33pijt7l9r356m0nannhjvsqsm2ndmbp2rpti/family1/87bc421d46b34b279e223df3e87c69eb 2023-03-08 07:55:40,625 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 07:55:40,626 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:40,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(1912): e9fb34d1dad0ba5071bdda7f618f683e/family1 is initiating minor compaction (all files) 2023-03-08 07:55:40,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2413429b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-08 07:55:40,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:40,628 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023030874eb312584654a41b1f66a82701f4e74 2023-03-08 07:55:40,628 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,629 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:40,629 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303088d7268d22609428a907a86f858e878eb 2023-03-08 07:55:40,630 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303082b49222f8228452a844bae45fb531d15 2023-03-08 07:55:40,630 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308d98d2ae599654d118bb3154758edfed6 2023-03-08 07:55:40,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 07:55:40,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(1912): a50cf6d148fcdc856ce86b2cdca8ceb8/family1 is initiating minor compaction (all files) 2023-03-08 07:55:40,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4c5e9aa7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-08 07:55:40,631 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030867641f1e3c3343dfa02b9d07bede8947 2023-03-08 07:55:40,632 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308dce94be4e5d84e58a45f40f06ae85cfe 2023-03-08 07:55:40,632 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 07:55:40,632 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 07:55:40,632 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:40,633 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308b2e993e56b4c47ddac7b2c7eb1f52063 2023-03-08 07:55:40,634 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308c5fb8c538efe4d7fb527af5c654043a2 2023-03-08 07:55:40,634 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 07:55:40,634 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 07:55:40,635 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:40,637 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,638 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:40,639 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family1/6adfa3060ff7434eac677c45520416ba as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/6adfa3060ff7434eac677c45520416ba 2023-03-08 07:55:40,640 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308496b41fed01d480e88cf9782d1a9441b 2023-03-08 07:55:40,642 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303084c385dedc0a147a4bcc4e0b141f9789a 2023-03-08 07:55:40,644 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/.tmp/family2/1901e05970da46738cfcba5d0e0ecc7d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/1901e05970da46738cfcba5d0e0ecc7d 2023-03-08 07:55:40,644 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308b4f121f78e5744b28982e6e085c607b5 2023-03-08 07:55:40,645 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 07:55:40,645 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308ba919832891b4d6fa81ba88a66caaf2b 2023-03-08 07:55:40,646 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 07:55:40,646 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 07:55:40,646 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 2fb06f0faeeffea363aea80614ef1a51/family1 of 2fb06f0faeeffea363aea80614ef1a51 into 6adfa3060ff7434eac677c45520416ba(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:40,646 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:40,646 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51., storeName=2fb06f0faeeffea363aea80614ef1a51/family1, priority=12, startTime=1678262139359; duration=1sec 2023-03-08 07:55:40,646 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-08 07:55:40,646 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 07:55:40,646 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e9fb34d1dad0ba5071bdda7f618f683e/family1 in testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:40,646 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 07:55:40,646 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-04fda751d9ed4b71bdfbd09e53c14b7b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-32ba1adf9a024898b96e4d8e4abf926c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-f32fa60784ae4ddabafe1a8293029119, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-97e4fcf5f5f6408f8f0653dcd03bc9dc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp, totalSize=28.9 K 2023-03-08 07:55:40,649 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:40,650 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-04fda751d9ed4b71bdfbd09e53c14b7b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116959 2023-03-08 07:55:40,653 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-32ba1adf9a024898b96e4d8e4abf926c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:40,654 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-f32fa60784ae4ddabafe1a8293029119, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:40,654 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-97e4fcf5f5f6408f8f0653dcd03bc9dc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:40,655 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:40,655 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ce5bd526acbb49409eea2ccb873e7884_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678262116959 2023-03-08 07:55:40,657 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 2fb06f0faeeffea363aea80614ef1a51/family2 of 2fb06f0faeeffea363aea80614ef1a51 into 1901e05970da46738cfcba5d0e0ecc7d(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:40,658 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:55:40,658 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51., storeName=2fb06f0faeeffea363aea80614ef1a51/family2, priority=12, startTime=1678262139359; duration=1sec 2023-03-08 07:55:40,658 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-08 07:55:40,658 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a50cf6d148fcdc856ce86b2cdca8ceb8/family1 in testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:40,658 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-8986c97ff59f4b56bf600bf2a1520ee7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-deb2ab70953849c584ea0ec88bf3ed81, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f8a89224d8404fd3b8cec9768b68df23, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-57716fd64aec4b8a9f14ced6989e1725, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp, totalSize=28.9 K 2023-03-08 07:55:40,660 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:40,661 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-8986c97ff59f4b56bf600bf2a1520ee7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116957 2023-03-08 07:55:40,662 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-deb2ab70953849c584ea0ec88bf3ed81, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:40,663 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:40,663 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f8a89224d8404fd3b8cec9768b68df23, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120561 2023-03-08 07:55:40,664 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-57716fd64aec4b8a9f14ced6989e1725, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121978 2023-03-08 07:55:40,664 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting f6730f2c27674b7293b9952927299595_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678262116957 2023-03-08 07:55:40,669 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:40,670 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:40,685 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): e9fb34d1dad0ba5071bdda7f618f683e#family1#compaction#115 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:40,689 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=284 (was 244) - Thread LEAK? -, OpenFileDescriptor=764 (was 622) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=187 (was 199), ProcessCount=170 (was 170), AvailableMemoryMB=7452 (was 7554) 2023-03-08 07:55:40,695 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] throttle.PressureAwareThroughputController(145): a50cf6d148fcdc856ce86b2cdca8ceb8#family1#compaction#116 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:40,706 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=290, OpenFileDescriptor=768, MaxFileDescriptor=60000, SystemLoadAverage=187, ProcessCount=170, AvailableMemoryMB=7452 2023-03-08 07:55:40,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-08 07:55:40,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-08 07:55:40,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 07:55:40,719 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 07:55:40,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 12 msec 2023-03-08 07:55:40,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-b1a0e74efa4c4846bc9024a6de7e8074, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-4d800d09acb843c68c40907b9749c4b0, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-5e9ed625067b4b08a3f7b952e348a0b7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-c15382d6134c492ab93f119690918ab1] to archive 2023-03-08 07:55:40,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:40,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-b1a0e74efa4c4846bc9024a6de7e8074 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-b1a0e74efa4c4846bc9024a6de7e8074 2023-03-08 07:55:40,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-4d800d09acb843c68c40907b9749c4b0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-4d800d09acb843c68c40907b9749c4b0 2023-03-08 07:55:40,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-5e9ed625067b4b08a3f7b952e348a0b7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-5e9ed625067b4b08a3f7b952e348a0b7 2023-03-08 07:55:40,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-c15382d6134c492ab93f119690918ab1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family1/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-c15382d6134c492ab93f119690918ab1 2023-03-08 07:55:40,747 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-650c0bf692e9431184c7ef97d0ef75ea, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-bec62040e49a41f3892006664123aba7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-6f57e6ccdff045cd8423070fb969e229, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-ae1cf04043de49c884f5504f54ee4297] to archive 2023-03-08 07:55:40,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:40,749 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-650c0bf692e9431184c7ef97d0ef75ea to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-650c0bf692e9431184c7ef97d0ef75ea 2023-03-08 07:55:40,750 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-bec62040e49a41f3892006664123aba7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-bec62040e49a41f3892006664123aba7 2023-03-08 07:55:40,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-6f57e6ccdff045cd8423070fb969e229 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-6f57e6ccdff045cd8423070fb969e229 2023-03-08 07:55:40,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-ae1cf04043de49c884f5504f54ee4297 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/family2/testCompactionWithHFileLink=2fb06f0faeeffea363aea80614ef1a51-ae1cf04043de49c884f5504f54ee4297 2023-03-08 07:55:40,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 07:55:40,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 07:55:40,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-08 07:55:40,822 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:55:40,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-08 07:55:40,823 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:55:40,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 07:55:40,824 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:40,824 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635 2023-03-08 07:55:40,824 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:40,825 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790 empty. 2023-03-08 07:55:40,825 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635 empty. 2023-03-08 07:55:40,825 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603 empty. 2023-03-08 07:55:40,825 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:40,825 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:40,825 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635 2023-03-08 07:55:40,825 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-08 07:55:40,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 07:55:41,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 07:55:41,232 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 07:55:41,233 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => f3f6a8571aa8d2b03ba727aae5b43603, NAME => 'ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:41,233 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 05308de2ff490d772d5a663e6692c790, NAME => 'ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:41,234 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => eba46d84043812856e232d435ccca635, NAME => 'ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:41,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 07:55:41,517 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family1/4ebfd760df58437c916be064ca1cdda2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/4ebfd760df58437c916be064ca1cdda2 2023-03-08 07:55:41,522 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in a50cf6d148fcdc856ce86b2cdca8ceb8/family1 of a50cf6d148fcdc856ce86b2cdca8ceb8 into 4ebfd760df58437c916be064ca1cdda2(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:55:41,522 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:41,522 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678262140631; duration=0sec 2023-03-08 07:55:41,522 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 07:55:41,522 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 07:55:41,524 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 07:55:41,524 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1912): e9fb34d1dad0ba5071bdda7f618f683e/family2 is initiating minor compaction (all files) 2023-03-08 07:55:41,524 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e9fb34d1dad0ba5071bdda7f618f683e/family2 in testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:55:41,524 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9cae59daa977464dbc7c2d8b8a2bed4a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-7e24d05a9202460e924485dec0ce3271, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-b6c0e39cb8fd4b39a6570ccd11d3b508, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9c31cc68bf324de28df80734eb1c30e2] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp, totalSize=20.4 K 2023-03-08 07:55:41,525 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9cae59daa977464dbc7c2d8b8a2bed4a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116959 2023-03-08 07:55:41,526 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-7e24d05a9202460e924485dec0ce3271, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:41,526 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-b6c0e39cb8fd4b39a6570ccd11d3b508, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120562 2023-03-08 07:55:41,527 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9c31cc68bf324de28df80734eb1c30e2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121979 2023-03-08 07:55:41,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-8986c97ff59f4b56bf600bf2a1520ee7, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-deb2ab70953849c584ea0ec88bf3ed81, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f8a89224d8404fd3b8cec9768b68df23, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-57716fd64aec4b8a9f14ced6989e1725, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_] to archive 2023-03-08 07:55:41,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:41,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-8986c97ff59f4b56bf600bf2a1520ee7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-8986c97ff59f4b56bf600bf2a1520ee7 2023-03-08 07:55:41,539 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] throttle.PressureAwareThroughputController(145): e9fb34d1dad0ba5071bdda7f618f683e#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:41,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-deb2ab70953849c584ea0ec88bf3ed81 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-deb2ab70953849c584ea0ec88bf3ed81 2023-03-08 07:55:41,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f8a89224d8404fd3b8cec9768b68df23 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f8a89224d8404fd3b8cec9768b68df23 2023-03-08 07:55:41,544 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-57716fd64aec4b8a9f14ced6989e1725 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-57716fd64aec4b8a9f14ced6989e1725 2023-03-08 07:55:41,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family1/f6730f2c27674b7293b9952927299595_SeqId_34_ 2023-03-08 07:55:41,576 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 05308de2ff490d772d5a663e6692c790, disabling compactions & flushes 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:41,640 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing eba46d84043812856e232d435ccca635, disabling compactions & flushes 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing f3f6a8571aa8d2b03ba727aae5b43603, disabling compactions & flushes 2023-03-08 07:55:41,640 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. after waiting 0 ms 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. after waiting 0 ms 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:41,640 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,640 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:41,640 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:41,640 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. after waiting 0 ms 2023-03-08 07:55:41,641 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,641 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,641 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:41,642 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:55:41,643 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678262141643"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262141643"}]},"ts":"1678262141643"} 2023-03-08 07:55:41,643 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678262141643"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262141643"}]},"ts":"1678262141643"} 2023-03-08 07:55:41,643 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678262141643"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262141643"}]},"ts":"1678262141643"} 2023-03-08 07:55:41,645 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 07:55:41,646 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:55:41,646 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262141646"}]},"ts":"1678262141646"} 2023-03-08 07:55:41,647 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-08 07:55:41,650 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f3f6a8571aa8d2b03ba727aae5b43603, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=eba46d84043812856e232d435ccca635, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05308de2ff490d772d5a663e6692c790, ASSIGN}] 2023-03-08 07:55:41,651 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=eba46d84043812856e232d435ccca635, ASSIGN 2023-03-08 07:55:41,651 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f3f6a8571aa8d2b03ba727aae5b43603, ASSIGN 2023-03-08 07:55:41,651 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05308de2ff490d772d5a663e6692c790, ASSIGN 2023-03-08 07:55:41,652 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=eba46d84043812856e232d435ccca635, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:41,652 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05308de2ff490d772d5a663e6692c790, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:41,652 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f3f6a8571aa8d2b03ba727aae5b43603, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:41,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=f3f6a8571aa8d2b03ba727aae5b43603, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:41,804 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=eba46d84043812856e232d435ccca635, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:41,804 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=05308de2ff490d772d5a663e6692c790, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:41,805 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678262141804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262141804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262141804"}]},"ts":"1678262141804"} 2023-03-08 07:55:41,805 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678262141804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262141804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262141804"}]},"ts":"1678262141804"} 2023-03-08 07:55:41,805 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678262141804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262141804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262141804"}]},"ts":"1678262141804"} 2023-03-08 07:55:41,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure f3f6a8571aa8d2b03ba727aae5b43603, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:41,807 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure eba46d84043812856e232d435ccca635, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:41,807 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 05308de2ff490d772d5a663e6692c790, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:41,911 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family1/39d145e062b14c7ba619404e4d0bd2dd as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/39d145e062b14c7ba619404e4d0bd2dd 2023-03-08 07:55:41,915 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in e9fb34d1dad0ba5071bdda7f618f683e/family1 of e9fb34d1dad0ba5071bdda7f618f683e into 39d145e062b14c7ba619404e4d0bd2dd(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:41,915 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:41,915 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678262140626; duration=1sec 2023-03-08 07:55:41,915 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 07:55:41,915 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 07:55:41,917 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 07:55:41,917 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1912): a50cf6d148fcdc856ce86b2cdca8ceb8/family2 is initiating minor compaction (all files) 2023-03-08 07:55:41,917 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a50cf6d148fcdc856ce86b2cdca8ceb8/family2 in testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:55:41,917 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f46ce0a8709142a28bb3626c172a4a46, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-7ff98be37e5643c59927680a6da57fdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-c33d6be82d7e460db1fdbbf7131c6545, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-4b19bbe0117d46e4937560e8cb137e61] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp, totalSize=20.4 K 2023-03-08 07:55:41,917 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f46ce0a8709142a28bb3626c172a4a46, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262116957 2023-03-08 07:55:41,918 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-7ff98be37e5643c59927680a6da57fdc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262118749 2023-03-08 07:55:41,918 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-c33d6be82d7e460db1fdbbf7131c6545, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262120561 2023-03-08 07:55:41,919 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-4b19bbe0117d46e4937560e8cb137e61, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262121978 2023-03-08 07:55:41,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 07:55:41,931 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): a50cf6d148fcdc856ce86b2cdca8ceb8#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:41,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-04fda751d9ed4b71bdfbd09e53c14b7b, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-32ba1adf9a024898b96e4d8e4abf926c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-f32fa60784ae4ddabafe1a8293029119, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-97e4fcf5f5f6408f8f0653dcd03bc9dc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_] to archive 2023-03-08 07:55:41,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:41,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-04fda751d9ed4b71bdfbd09e53c14b7b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-04fda751d9ed4b71bdfbd09e53c14b7b 2023-03-08 07:55:41,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-32ba1adf9a024898b96e4d8e4abf926c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-32ba1adf9a024898b96e4d8e4abf926c 2023-03-08 07:55:41,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-f32fa60784ae4ddabafe1a8293029119 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-f32fa60784ae4ddabafe1a8293029119 2023-03-08 07:55:41,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-97e4fcf5f5f6408f8f0653dcd03bc9dc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-97e4fcf5f5f6408f8f0653dcd03bc9dc 2023-03-08 07:55:41,949 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family1/ce5bd526acbb49409eea2ccb873e7884_SeqId_34_ 2023-03-08 07:55:41,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f3f6a8571aa8d2b03ba727aae5b43603, NAME => 'ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 07:55:41,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:41,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,962 INFO [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,963 DEBUG [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1 2023-03-08 07:55:41,963 DEBUG [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1 2023-03-08 07:55:41,963 INFO [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3f6a8571aa8d2b03ba727aae5b43603 columnFamilyName family1 2023-03-08 07:55:41,964 INFO [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] regionserver.HStore(310): Store=f3f6a8571aa8d2b03ba727aae5b43603/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:41,965 INFO [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,966 DEBUG [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2 2023-03-08 07:55:41,966 DEBUG [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2 2023-03-08 07:55:41,966 INFO [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3f6a8571aa8d2b03ba727aae5b43603 columnFamilyName family2 2023-03-08 07:55:41,967 INFO [StoreOpener-f3f6a8571aa8d2b03ba727aae5b43603-1] regionserver.HStore(310): Store=f3f6a8571aa8d2b03ba727aae5b43603/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:41,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:41,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:55:41,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:41,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f3f6a8571aa8d2b03ba727aae5b43603; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72598109, jitterRate=0.08179612457752228}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:41,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:41,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603., pid=109, masterSystemTime=1678262141958 2023-03-08 07:55:41,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:41,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eba46d84043812856e232d435ccca635, NAME => 'ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 07:55:41,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:41,974 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=f3f6a8571aa8d2b03ba727aae5b43603, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:41,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,975 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678262141974"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262141974"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262141974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262141974"}]},"ts":"1678262141974"} 2023-03-08 07:55:41,976 INFO [StoreOpener-eba46d84043812856e232d435ccca635-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,977 DEBUG [StoreOpener-eba46d84043812856e232d435ccca635-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1 2023-03-08 07:55:41,977 DEBUG [StoreOpener-eba46d84043812856e232d435ccca635-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1 2023-03-08 07:55:41,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-03-08 07:55:41,977 INFO [StoreOpener-eba46d84043812856e232d435ccca635-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eba46d84043812856e232d435ccca635 columnFamilyName family1 2023-03-08 07:55:41,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure f3f6a8571aa8d2b03ba727aae5b43603, server=jenkins-hbase4.apache.org,33669,1678262016084 in 170 msec 2023-03-08 07:55:41,978 INFO [StoreOpener-eba46d84043812856e232d435ccca635-1] regionserver.HStore(310): Store=eba46d84043812856e232d435ccca635/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:41,978 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f3f6a8571aa8d2b03ba727aae5b43603, ASSIGN in 328 msec 2023-03-08 07:55:41,978 INFO [StoreOpener-eba46d84043812856e232d435ccca635-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,979 DEBUG [StoreOpener-eba46d84043812856e232d435ccca635-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2 2023-03-08 07:55:41,979 DEBUG [StoreOpener-eba46d84043812856e232d435ccca635-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2 2023-03-08 07:55:41,979 INFO [StoreOpener-eba46d84043812856e232d435ccca635-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eba46d84043812856e232d435ccca635 columnFamilyName family2 2023-03-08 07:55:41,980 INFO [StoreOpener-eba46d84043812856e232d435ccca635-1] regionserver.HStore(310): Store=eba46d84043812856e232d435ccca635/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:41,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:41,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eba46d84043812856e232d435ccca635 2023-03-08 07:55:41,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:41,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eba46d84043812856e232d435ccca635; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62427712, jitterRate=-0.06975460052490234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:41,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:41,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635., pid=110, masterSystemTime=1678262141958 2023-03-08 07:55:41,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:41,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:41,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 05308de2ff490d772d5a663e6692c790, NAME => 'ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 07:55:41,988 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=eba46d84043812856e232d435ccca635, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:41,988 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678262141988"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262141988"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262141988"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262141988"}]},"ts":"1678262141988"} 2023-03-08 07:55:41,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:41,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,989 INFO [StoreOpener-05308de2ff490d772d5a663e6692c790-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,990 DEBUG [StoreOpener-05308de2ff490d772d5a663e6692c790-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1 2023-03-08 07:55:41,990 DEBUG [StoreOpener-05308de2ff490d772d5a663e6692c790-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1 2023-03-08 07:55:41,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-03-08 07:55:41,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure eba46d84043812856e232d435ccca635, server=jenkins-hbase4.apache.org,33669,1678262016084 in 182 msec 2023-03-08 07:55:41,991 INFO [StoreOpener-05308de2ff490d772d5a663e6692c790-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 05308de2ff490d772d5a663e6692c790 columnFamilyName family1 2023-03-08 07:55:41,991 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=eba46d84043812856e232d435ccca635, ASSIGN in 341 msec 2023-03-08 07:55:41,991 INFO [StoreOpener-05308de2ff490d772d5a663e6692c790-1] regionserver.HStore(310): Store=05308de2ff490d772d5a663e6692c790/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:41,992 INFO [StoreOpener-05308de2ff490d772d5a663e6692c790-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,993 DEBUG [StoreOpener-05308de2ff490d772d5a663e6692c790-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2 2023-03-08 07:55:41,993 DEBUG [StoreOpener-05308de2ff490d772d5a663e6692c790-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2 2023-03-08 07:55:41,993 INFO [StoreOpener-05308de2ff490d772d5a663e6692c790-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 05308de2ff490d772d5a663e6692c790 columnFamilyName family2 2023-03-08 07:55:41,994 INFO [StoreOpener-05308de2ff490d772d5a663e6692c790-1] regionserver.HStore(310): Store=05308de2ff490d772d5a663e6692c790/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:41,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:55:41,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 05308de2ff490d772d5a663e6692c790 2023-03-08 07:55:41,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:41,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 05308de2ff490d772d5a663e6692c790; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69569216, jitterRate=0.03666210174560547}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:55:41,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:41,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790., pid=111, masterSystemTime=1678262141958 2023-03-08 07:55:42,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:42,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:42,001 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=05308de2ff490d772d5a663e6692c790, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:42,001 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678262142001"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262142001"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262142001"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262142001"}]},"ts":"1678262142001"} 2023-03-08 07:55:42,003 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-03-08 07:55:42,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 05308de2ff490d772d5a663e6692c790, server=jenkins-hbase4.apache.org,33669,1678262016084 in 195 msec 2023-03-08 07:55:42,005 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-03-08 07:55:42,005 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05308de2ff490d772d5a663e6692c790, ASSIGN in 354 msec 2023-03-08 07:55:42,005 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:55:42,006 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262142006"}]},"ts":"1678262142006"} 2023-03-08 07:55:42,007 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-08 07:55:42,009 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:55:42,010 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1890 sec 2023-03-08 07:55:42,356 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/.tmp/family2/20c8978c5d9a4c1bac00e3b09befd492 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/20c8978c5d9a4c1bac00e3b09befd492 2023-03-08 07:55:42,360 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in e9fb34d1dad0ba5071bdda7f618f683e/family2 of e9fb34d1dad0ba5071bdda7f618f683e into 20c8978c5d9a4c1bac00e3b09befd492(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:55:42,360 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:55:42,360 INFO [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e., storeName=e9fb34d1dad0ba5071bdda7f618f683e/family2, priority=12, startTime=1678262139451; duration=0sec 2023-03-08 07:55:42,360 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 07:55:42,360 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 07:55:42,360 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 07:55:42,360 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 07:55:42,360 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. because compaction request was cancelled 2023-03-08 07:55:42,361 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 07:55:42,361 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 07:55:42,361 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 07:55:42,361 DEBUG [RS:0;jenkins-hbase4:33669-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. because compaction request was cancelled 2023-03-08 07:55:42,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9cae59daa977464dbc7c2d8b8a2bed4a, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-7e24d05a9202460e924485dec0ce3271, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-b6c0e39cb8fd4b39a6570ccd11d3b508, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9c31cc68bf324de28df80734eb1c30e2] to archive 2023-03-08 07:55:42,437 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:42,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9cae59daa977464dbc7c2d8b8a2bed4a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9cae59daa977464dbc7c2d8b8a2bed4a 2023-03-08 07:55:42,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-7e24d05a9202460e924485dec0ce3271 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-7e24d05a9202460e924485dec0ce3271 2023-03-08 07:55:42,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-b6c0e39cb8fd4b39a6570ccd11d3b508 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-b6c0e39cb8fd4b39a6570ccd11d3b508 2023-03-08 07:55:42,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9c31cc68bf324de28df80734eb1c30e2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/family2/testCompactionWithHFileLink=e9fb34d1dad0ba5071bdda7f618f683e-9c31cc68bf324de28df80734eb1c30e2 2023-03-08 07:55:42,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 07:55:42,928 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-08 07:55:42,934 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:42,934 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:42,934 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:42,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:42,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 07:55:42,938 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 07:55:42,938 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:42,938 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 07:55:42,938 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 07:55:42,938 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:42,938 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:42,942 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:42,942 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:42,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:42,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:42,942 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:42,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:42,942 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:42,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:42,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:42,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:42,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 07:55:42,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 07:55:42,943 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 07:55:42,943 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:42,943 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 07:55:42,944 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:42,944 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. started... 2023-03-08 07:55:42,944 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:42,944 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:42,945 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eba46d84043812856e232d435ccca635 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:55:42,945 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:42,945 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:42,946 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. started... 2023-03-08 07:55:42,944 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. started... 2023-03-08 07:55:42,946 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3f6a8571aa8d2b03ba727aae5b43603 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:55:42,946 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 05308de2ff490d772d5a663e6692c790 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:55:43,147 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/.tmp/family2/dc0846637ce745d6b32bb57818991d46 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/dc0846637ce745d6b32bb57818991d46 2023-03-08 07:55:43,151 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in a50cf6d148fcdc856ce86b2cdca8ceb8/family2 of a50cf6d148fcdc856ce86b2cdca8ceb8 into dc0846637ce745d6b32bb57818991d46(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:43,151 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:55:43,151 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8., storeName=a50cf6d148fcdc856ce86b2cdca8ceb8/family2, priority=12, startTime=1678262139516; duration=1sec 2023-03-08 07:55:43,151 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:55:43,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f46ce0a8709142a28bb3626c172a4a46, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-7ff98be37e5643c59927680a6da57fdc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-c33d6be82d7e460db1fdbbf7131c6545, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-4b19bbe0117d46e4937560e8cb137e61] to archive 2023-03-08 07:55:43,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:43,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f46ce0a8709142a28bb3626c172a4a46 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-f46ce0a8709142a28bb3626c172a4a46 2023-03-08 07:55:43,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-7ff98be37e5643c59927680a6da57fdc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-7ff98be37e5643c59927680a6da57fdc 2023-03-08 07:55:43,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-c33d6be82d7e460db1fdbbf7131c6545 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-c33d6be82d7e460db1fdbbf7131c6545 2023-03-08 07:55:43,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-4b19bbe0117d46e4937560e8cb137e61 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/family2/testCompactionWithHFileLink=a50cf6d148fcdc856ce86b2cdca8ceb8-4b19bbe0117d46e4937560e8cb137e61 2023-03-08 07:55:43,357 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308ed0fce9b081047eab78288255f1076aa to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ed0fce9b081047eab78288255f1076aa 2023-03-08 07:55:43,362 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303088a7c9708a2bd40a694b83f827af975a8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303088a7c9708a2bd40a694b83f827af975a8 2023-03-08 07:55:43,362 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030877346283e2b34d49a334294b9ea98d76 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030877346283e2b34d49a334294b9ea98d76 2023-03-08 07:55:43,367 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/caddd693ceba4cf9b09c9d83337673b5 2023-03-08 07:55:43,369 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/f84b2102c87d48279dae98721103119e 2023-03-08 07:55:43,762 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/82448cd237fe493a9d7832b3407957c1 2023-03-08 07:55:43,782 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a954dc675f274d9ba5513c624072c49d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308a954dc675f274d9ba5513c624072c49d 2023-03-08 07:55:43,783 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030894a30e00401e404e9ed12d8bc7f7de9e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030894a30e00401e404e9ed12d8bc7f7de9e 2023-03-08 07:55:44,098 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-08 07:55:44,176 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308f233907fba9a42a2bde7e9ae0578265e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308f233907fba9a42a2bde7e9ae0578265e 2023-03-08 07:55:44,186 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/1dfacb090cde4b73b26aa687d5af0e77 2023-03-08 07:55:44,187 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/aaebaf6e905541098ebc7b7104e4ee60 2023-03-08 07:55:44,190 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/caddd693ceba4cf9b09c9d83337673b5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/caddd693ceba4cf9b09c9d83337673b5 2023-03-08 07:55:44,191 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/f84b2102c87d48279dae98721103119e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f84b2102c87d48279dae98721103119e 2023-03-08 07:55:44,194 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/caddd693ceba4cf9b09c9d83337673b5, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 07:55:44,195 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/1dfacb090cde4b73b26aa687d5af0e77 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/1dfacb090cde4b73b26aa687d5af0e77 2023-03-08 07:55:44,195 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f84b2102c87d48279dae98721103119e, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 07:55:44,195 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/aaebaf6e905541098ebc7b7104e4ee60 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/aaebaf6e905541098ebc7b7104e4ee60 2023-03-08 07:55:44,198 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/1dfacb090cde4b73b26aa687d5af0e77, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:55:44,199 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/aaebaf6e905541098ebc7b7104e4ee60, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:55:44,199 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f3f6a8571aa8d2b03ba727aae5b43603 in 1253ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:44,199 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for eba46d84043812856e232d435ccca635 in 1254ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:44,200 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:44,200 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:44,200 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:44,200 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:44,200 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:44,200 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:44,580 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/e80e0b25f8bd4ed98968f97da77e7665 2023-03-08 07:55:44,585 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/82448cd237fe493a9d7832b3407957c1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/82448cd237fe493a9d7832b3407957c1 2023-03-08 07:55:44,588 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/82448cd237fe493a9d7832b3407957c1, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 07:55:44,589 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/e80e0b25f8bd4ed98968f97da77e7665 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/e80e0b25f8bd4ed98968f97da77e7665 2023-03-08 07:55:44,592 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/e80e0b25f8bd4ed98968f97da77e7665, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:55:44,593 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 05308de2ff490d772d5a663e6692c790 in 1647ms, sequenceid=5, compaction requested=false 2023-03-08 07:55:44,593 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:44,593 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:44,593 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:44,593 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:44,593 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,594 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 07:55:44,594 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 07:55:44,595 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:44,595 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,596 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:44,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 07:55:44,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cd9e568[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:44,597 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 07:55:44,597 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,598 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,598 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 07:55:44,598 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 07:55:44,598 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:44,598 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,598 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:44,599 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:44,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:44,599 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 07:55:44,599 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:44,599 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 07:55:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:44,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:44,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:44,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 07:55:44,601 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 07:55:44,601 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:44,601 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:44,602 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 07:55:44,602 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:44,604 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,604 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,604 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:44,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:44,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:44,604 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:44,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:44,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:44,606 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,608 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:44,608 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:44,608 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 07:55:44,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:44,608 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:44,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:44,608 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:44,609 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:44,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:44,608 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:44,609 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:44,609 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,609 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,609 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:44,609 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:44,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:44,709 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:44,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:44,711 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:44,711 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:44,711 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:44,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:44,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:44,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 07:55:44,715 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 07:55:44,715 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:44,715 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 07:55:44,715 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 07:55:44,715 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,715 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:44,716 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,717 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:44,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:44,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:44,717 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:44,717 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:44,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:44,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:44,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:44,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:44,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 07:55:44,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 07:55:44,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 07:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 07:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 07:55:44,718 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:44,718 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:44,718 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. started... 2023-03-08 07:55:44,718 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:44,718 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. started... 2023-03-08 07:55:44,718 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. started... 2023-03-08 07:55:44,718 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eba46d84043812856e232d435ccca635 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:44,718 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3f6a8571aa8d2b03ba727aae5b43603 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:55:44,718 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 05308de2ff490d772d5a663e6692c790 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:55:44,731 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087c804a090c62459aa243a5b92be244e9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303087c804a090c62459aa243a5b92be244e9 2023-03-08 07:55:44,736 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/fc048389f3924791a003d6ea27f2108c 2023-03-08 07:55:44,748 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030858fbba2a6d5c406dae9668fa677da21e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023030858fbba2a6d5c406dae9668fa677da21e 2023-03-08 07:55:44,752 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/34f688040c7e4dbcb6d0f54a2a9efe9f 2023-03-08 07:55:44,755 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/fc048389f3924791a003d6ea27f2108c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/fc048389f3924791a003d6ea27f2108c 2023-03-08 07:55:44,759 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/fc048389f3924791a003d6ea27f2108c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 07:55:44,759 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/34f688040c7e4dbcb6d0f54a2a9efe9f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/34f688040c7e4dbcb6d0f54a2a9efe9f 2023-03-08 07:55:44,762 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/34f688040c7e4dbcb6d0f54a2a9efe9f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:55:44,763 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 05308de2ff490d772d5a663e6692c790 in 45ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:44,763 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:44,763 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:44,764 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:45,130 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e99373a0441e4a62ae02bc63acabe115 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308e99373a0441e4a62ae02bc63acabe115 2023-03-08 07:55:45,130 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085544ad4e7b7f4f15bdaf2819be0060de to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303085544ad4e7b7f4f15bdaf2819be0060de 2023-03-08 07:55:45,535 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/8cac38229dad41aca20934d10c534a15 2023-03-08 07:55:45,535 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/1b30e136f26c4d629e6bf412e85334bd 2023-03-08 07:55:45,952 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308f82868962f144952b615edb126621515 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308f82868962f144952b615edb126621515 2023-03-08 07:55:45,958 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303082750867cbc3047be8b0592da78ebeaac to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303082750867cbc3047be8b0592da78ebeaac 2023-03-08 07:55:46,358 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/c451e5661c5a49619ca4b14a8e35c327 2023-03-08 07:55:46,362 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/0d7f9e02a7964211a2dc4ee9907a66c8 2023-03-08 07:55:46,362 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/8cac38229dad41aca20934d10c534a15 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/8cac38229dad41aca20934d10c534a15 2023-03-08 07:55:46,366 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/8cac38229dad41aca20934d10c534a15, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 07:55:46,366 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/1b30e136f26c4d629e6bf412e85334bd as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/1b30e136f26c4d629e6bf412e85334bd 2023-03-08 07:55:46,367 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/c451e5661c5a49619ca4b14a8e35c327 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/c451e5661c5a49619ca4b14a8e35c327 2023-03-08 07:55:46,370 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/1b30e136f26c4d629e6bf412e85334bd, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 07:55:46,370 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/c451e5661c5a49619ca4b14a8e35c327, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:55:46,371 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/0d7f9e02a7964211a2dc4ee9907a66c8 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/0d7f9e02a7964211a2dc4ee9907a66c8 2023-03-08 07:55:46,371 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for eba46d84043812856e232d435ccca635 in 1653ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:46,371 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:46,371 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:46,371 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:46,374 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/0d7f9e02a7964211a2dc4ee9907a66c8, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:55:46,375 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f3f6a8571aa8d2b03ba727aae5b43603 in 1657ms, sequenceid=9, compaction requested=false 2023-03-08 07:55:46,375 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:46,375 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 07:55:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 07:55:46,377 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,377 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:46,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:46,377 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,377 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:46,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:46,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:46,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:46,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 07:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e3bb493[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:46,379 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 07:55:46,379 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,381 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,381 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 07:55:46,381 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 07:55:46,381 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:46,381 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,381 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:46,385 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,385 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 07:55:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:46,385 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:46,385 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 07:55:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 07:55:46,387 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 07:55:46,387 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:46,387 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:46,387 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 07:55:46,387 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:46,389 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,389 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:46,389 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,389 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:46,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:46,390 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,393 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,393 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:46,393 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:46,393 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:46,394 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:46,393 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 07:55:46,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:46,394 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:46,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 07:55:46,394 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,394 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:46,394 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,394 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:46,394 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:46,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:46,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:46,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:46,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:46,494 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:46,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:46,496 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:46,496 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:46,497 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:46,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:46,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:46,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 07:55:46,501 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 07:55:46,501 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:46,501 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 07:55:46,501 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 07:55:46,501 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,502 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:46,503 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,503 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:46,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:46,503 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:46,503 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:46,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:46,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:46,504 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 07:55:46,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 07:55:46,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 07:55:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 07:55:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 07:55:46,504 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:46,504 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:46,504 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:46,504 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. started... 2023-03-08 07:55:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:46,504 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. started... 2023-03-08 07:55:46,505 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3f6a8571aa8d2b03ba727aae5b43603 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:55:46,504 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. started... 2023-03-08 07:55:46,505 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 05308de2ff490d772d5a663e6692c790 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:55:46,505 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eba46d84043812856e232d435ccca635 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:55:46,521 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ee33741d13eb477a86310ac5d242435a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308ee33741d13eb477a86310ac5d242435a 2023-03-08 07:55:46,524 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303086bee296e4b804979a2a0ec3a65c2b2ce to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303086bee296e4b804979a2a0ec3a65c2b2ce 2023-03-08 07:55:46,525 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303081a5e28e2092a4f3b967d7d2c6968d500 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303081a5e28e2092a4f3b967d7d2c6968d500 2023-03-08 07:55:46,527 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c 2023-03-08 07:55:46,532 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/25fb314d1fbd4e208992881777d246ec 2023-03-08 07:55:46,539 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b5e76841b527471cbee7991ea5f564e7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308b5e76841b527471cbee7991ea5f564e7 2023-03-08 07:55:46,932 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/b600621526da40e983b98ff6049a2896 2023-03-08 07:55:46,948 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085326bad7f77c48ea9acc3d2b4e72bcec to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303085326bad7f77c48ea9acc3d2b4e72bcec 2023-03-08 07:55:46,949 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/f760838205134a809589c32e455a5008 2023-03-08 07:55:46,953 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c 2023-03-08 07:55:46,956 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 07:55:46,957 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/f760838205134a809589c32e455a5008 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/f760838205134a809589c32e455a5008 2023-03-08 07:55:46,959 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/f760838205134a809589c32e455a5008, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:55:46,960 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f3f6a8571aa8d2b03ba727aae5b43603 in 456ms, sequenceid=13, compaction requested=true 2023-03-08 07:55:46,960 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:46,960 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:46,960 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:47,348 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308439d96baf94c41a093ab4f7fea47a5af to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308439d96baf94c41a093ab4f7fea47a5af 2023-03-08 07:55:47,353 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/d5cbf731831b4ba282a423f86b509911 2023-03-08 07:55:47,357 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/25fb314d1fbd4e208992881777d246ec as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/25fb314d1fbd4e208992881777d246ec 2023-03-08 07:55:47,360 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/25fb314d1fbd4e208992881777d246ec, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 07:55:47,361 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/d5cbf731831b4ba282a423f86b509911 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/d5cbf731831b4ba282a423f86b509911 2023-03-08 07:55:47,365 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/d5cbf731831b4ba282a423f86b509911, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:55:47,366 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 05308de2ff490d772d5a663e6692c790 in 861ms, sequenceid=13, compaction requested=true 2023-03-08 07:55:47,366 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:47,366 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:47,366 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:47,462 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:55:47,753 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/78d33d6ee2754ef0b4efa279c263120c 2023-03-08 07:55:47,757 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/b600621526da40e983b98ff6049a2896 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/b600621526da40e983b98ff6049a2896 2023-03-08 07:55:47,760 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/b600621526da40e983b98ff6049a2896, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 07:55:47,761 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/78d33d6ee2754ef0b4efa279c263120c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/78d33d6ee2754ef0b4efa279c263120c 2023-03-08 07:55:47,764 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/78d33d6ee2754ef0b4efa279c263120c, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:55:47,765 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for eba46d84043812856e232d435ccca635 in 1260ms, sequenceid=13, compaction requested=true 2023-03-08 07:55:47,765 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:47,765 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 07:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 07:55:47,766 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,766 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:47,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:47,766 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,767 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:47,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 07:55:47,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c7f78da[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:47,768 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 07:55:47,768 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 07:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 07:55:47,769 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,769 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:47,772 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 07:55:47,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:47,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 07:55:47,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:47,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 07:55:47,774 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 07:55:47,774 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:47,775 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,775 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:47,776 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:47,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:47,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:47,776 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,777 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:47,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:47,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 07:55:47,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:47,779 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:47,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:47,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:47,779 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:47,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:47,780 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:47,780 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,880 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:47,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:47,881 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:47,882 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:47,882 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:55:47,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:47,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:47,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 07:55:47,886 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 07:55:47,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:47,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 07:55:47,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 07:55:47,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:47,888 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:47,888 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:47,888 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:47,888 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:47,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:47,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 07:55:47,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 07:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 07:55:47,889 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 07:55:47,889 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:47,889 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 07:55:47,889 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:47,889 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:47,889 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:47,889 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:47,889 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:47,889 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. started... 2023-03-08 07:55:47,890 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. started... 2023-03-08 07:55:47,889 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. started... 2023-03-08 07:55:47,890 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eba46d84043812856e232d435ccca635 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:55:47,890 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 05308de2ff490d772d5a663e6692c790 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:55:47,890 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3f6a8571aa8d2b03ba727aae5b43603 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:55:48,302 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d014959cab1043fe995d94ba6fcfa133 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308d014959cab1043fe995d94ba6fcfa133 2023-03-08 07:55:48,304 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030867d2a1d32c28472c917507a4a7985eac to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030867d2a1d32c28472c917507a4a7985eac 2023-03-08 07:55:48,311 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030804c6f4ef67024aa5b814b9cbc89c2fce to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030804c6f4ef67024aa5b814b9cbc89c2fce 2023-03-08 07:55:48,711 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/7bd67b069a404818a9fbc8e14a79273c 2023-03-08 07:55:48,711 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/0f8e90762d7141dca4fa616c7d6e8202 2023-03-08 07:55:48,715 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/40351e4efd75494984f47ce9149c6ec9 2023-03-08 07:55:48,724 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030859a5eefa1c1a4feb9224ccc39a342462 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030859a5eefa1c1a4feb9224ccc39a342462 2023-03-08 07:55:49,126 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e8aa091e42144647942976cf0977e3ce to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308e8aa091e42144647942976cf0977e3ce 2023-03-08 07:55:49,127 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308353c199d760f4f24a68e43df553eb9a2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308353c199d760f4f24a68e43df553eb9a2 2023-03-08 07:55:49,128 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/af139451c42f4495b104381f2ac524f6 2023-03-08 07:55:49,131 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/8e6f7a6b544f44dbb52e5317b1b5e359 2023-03-08 07:55:49,132 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/7bd67b069a404818a9fbc8e14a79273c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/7bd67b069a404818a9fbc8e14a79273c 2023-03-08 07:55:49,135 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/40351e4efd75494984f47ce9149c6ec9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/40351e4efd75494984f47ce9149c6ec9 2023-03-08 07:55:49,136 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/7bd67b069a404818a9fbc8e14a79273c, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 07:55:49,136 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/af139451c42f4495b104381f2ac524f6 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/af139451c42f4495b104381f2ac524f6 2023-03-08 07:55:49,138 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/40351e4efd75494984f47ce9149c6ec9, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 07:55:49,138 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/8e6f7a6b544f44dbb52e5317b1b5e359 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/8e6f7a6b544f44dbb52e5317b1b5e359 2023-03-08 07:55:49,140 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/af139451c42f4495b104381f2ac524f6, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:55:49,140 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for eba46d84043812856e232d435ccca635 in 1250ms, sequenceid=17, compaction requested=true 2023-03-08 07:55:49,140 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:49,140 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:49,140 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:49,141 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/8e6f7a6b544f44dbb52e5317b1b5e359, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:55:49,142 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 05308de2ff490d772d5a663e6692c790 in 1252ms, sequenceid=17, compaction requested=true 2023-03-08 07:55:49,142 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:49,142 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:49,142 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:49,531 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/2c7b33be55634d719fe510366a9187a1 2023-03-08 07:55:49,535 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/0f8e90762d7141dca4fa616c7d6e8202 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/0f8e90762d7141dca4fa616c7d6e8202 2023-03-08 07:55:49,538 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/0f8e90762d7141dca4fa616c7d6e8202, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 07:55:49,539 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/2c7b33be55634d719fe510366a9187a1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/2c7b33be55634d719fe510366a9187a1 2023-03-08 07:55:49,543 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/2c7b33be55634d719fe510366a9187a1, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:55:49,543 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f3f6a8571aa8d2b03ba727aae5b43603 in 1653ms, sequenceid=17, compaction requested=true 2023-03-08 07:55:49,543 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:49,543 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:49,543 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:49,544 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:49,544 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,544 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 07:55:49,544 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 07:55:49,547 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:49,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:49,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 07:55:49,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2870fd11[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:49,548 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 07:55:49,548 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,549 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,550 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 07:55:49,550 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 07:55:49,550 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,550 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:49,550 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:49,551 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,551 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 07:55:49,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:49,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:49,551 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:49,552 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 07:55:49,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:49,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:49,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:49,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:49,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:49,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:49,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 07:55:49,553 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 07:55:49,553 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:49,553 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:49,553 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 07:55:49,553 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:49,555 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,555 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:49,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:49,555 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:49,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,555 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:49,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:49,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:49,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:49,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:49,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:49,556 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:49,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:49,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:49,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:49,559 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:49,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 07:55:49,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:49,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:49,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:49,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:49,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:49,559 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,560 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:49,560 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:49,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:49,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:49,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:49,560 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,660 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:49,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:49,671 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:49,681 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:49,688 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:49,689 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:49,703 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:49,703 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x680094d2 to 127.0.0.1:52387 2023-03-08 07:55:49,703 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:49,703 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:55:49,703 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33019f08 to 127.0.0.1:52387 2023-03-08 07:55:49,703 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:55:49,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:55:49,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 07:55:49,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 07:55:49,730 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 07:55:49,730 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:49,730 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 07:55:49,730 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 07:55:49,730 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,730 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:49,731 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,731 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:49,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:49,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:49,731 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:49,731 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:55:49,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:49,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:49,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:55:49,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:49,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 07:55:49,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 07:55:49,732 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 07:55:49,732 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:55:49,733 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 07:55:49,733 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:49,733 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. started... 2023-03-08 07:55:49,733 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eba46d84043812856e232d435ccca635 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:55:49,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:49,734 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:55:49,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:49,734 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:55:49,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. started... 2023-03-08 07:55:49,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. started... 2023-03-08 07:55:49,735 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 05308de2ff490d772d5a663e6692c790 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:55:49,735 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3f6a8571aa8d2b03ba727aae5b43603 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:55:50,555 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/ec407ac0a74e4d108620b311fc3d85a0 2023-03-08 07:55:50,555 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/1acdbd32e8c2449b9eda1d1f526721b5 2023-03-08 07:55:50,557 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/30d2f03b14d14dfe8066c44377577432 2023-03-08 07:55:50,559 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec407ac0a74e4d108620b311fc3d85a0 2023-03-08 07:55:50,561 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1acdbd32e8c2449b9eda1d1f526721b5 2023-03-08 07:55:50,562 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30d2f03b14d14dfe8066c44377577432 2023-03-08 07:55:50,971 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/589362f989e84566973123d297ef0e10 2023-03-08 07:55:50,976 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/ffdaa47005514b1cb74ba1c1358345e9 2023-03-08 07:55:50,976 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 589362f989e84566973123d297ef0e10 2023-03-08 07:55:50,977 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/1acdbd32e8c2449b9eda1d1f526721b5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/1acdbd32e8c2449b9eda1d1f526721b5 2023-03-08 07:55:50,979 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffdaa47005514b1cb74ba1c1358345e9 2023-03-08 07:55:50,980 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/30d2f03b14d14dfe8066c44377577432 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/30d2f03b14d14dfe8066c44377577432 2023-03-08 07:55:50,980 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1acdbd32e8c2449b9eda1d1f526721b5 2023-03-08 07:55:50,980 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/1acdbd32e8c2449b9eda1d1f526721b5, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 07:55:50,981 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/589362f989e84566973123d297ef0e10 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/589362f989e84566973123d297ef0e10 2023-03-08 07:55:50,984 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 589362f989e84566973123d297ef0e10 2023-03-08 07:55:50,984 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30d2f03b14d14dfe8066c44377577432 2023-03-08 07:55:50,984 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/589362f989e84566973123d297ef0e10, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 07:55:50,984 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/30d2f03b14d14dfe8066c44377577432, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 07:55:50,985 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 05308de2ff490d772d5a663e6692c790 in 1250ms, sequenceid=23, compaction requested=true 2023-03-08 07:55:50,985 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/ffdaa47005514b1cb74ba1c1358345e9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/ffdaa47005514b1cb74ba1c1358345e9 2023-03-08 07:55:50,985 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:50,985 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:50,985 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:55:50,989 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffdaa47005514b1cb74ba1c1358345e9 2023-03-08 07:55:50,989 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/ffdaa47005514b1cb74ba1c1358345e9, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 07:55:50,989 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for eba46d84043812856e232d435ccca635 in 1256ms, sequenceid=23, compaction requested=true 2023-03-08 07:55:50,989 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:50,989 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:50,989 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:55:51,375 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/b1ffcf35ca734467a7bb415d2b116fd4 2023-03-08 07:55:51,379 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1ffcf35ca734467a7bb415d2b116fd4 2023-03-08 07:55:51,379 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/ec407ac0a74e4d108620b311fc3d85a0 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/ec407ac0a74e4d108620b311fc3d85a0 2023-03-08 07:55:51,383 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec407ac0a74e4d108620b311fc3d85a0 2023-03-08 07:55:51,383 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/ec407ac0a74e4d108620b311fc3d85a0, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 07:55:51,383 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/b1ffcf35ca734467a7bb415d2b116fd4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/b1ffcf35ca734467a7bb415d2b116fd4 2023-03-08 07:55:51,387 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1ffcf35ca734467a7bb415d2b116fd4 2023-03-08 07:55:51,387 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/b1ffcf35ca734467a7bb415d2b116fd4, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 07:55:51,388 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f3f6a8571aa8d2b03ba727aae5b43603 in 1653ms, sequenceid=23, compaction requested=true 2023-03-08 07:55:51,388 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:51,388 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:51,388 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:55:51,388 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:55:51,388 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,388 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 07:55:51,388 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 07:55:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,392 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:51,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:55:51,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:51,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:51,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:51,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:51,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 07:55:51,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ba1afd2[Count = 0] remaining members to acquire global barrier 2023-03-08 07:55:51,394 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 07:55:51,394 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,395 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,395 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,395 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:55:51,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 07:55:51,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 07:55:51,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:55:51,398 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,398 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 07:55:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:51,398 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:51,398 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 07:55:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 07:55:51,400 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 07:55:51,400 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:55:51,400 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:55:51,400 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 07:55:51,400 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:55:51,403 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,403 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:55:51,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,403 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:51,403 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:51,404 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 07:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,406 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:55:51,406 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:55:51,407 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:55:51,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 07:55:51,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:55:51,406 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 07:55:51,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:55:51,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 07:55:51,407 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:55:51,507 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:55:51,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:55:51,509 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:51,509 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): eba46d84043812856e232d435ccca635/family1 is initiating major compaction (all files) 2023-03-08 07:55:51,509 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:51,509 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:51,510 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of eba46d84043812856e232d435ccca635/family1 in ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:51,510 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f84b2102c87d48279dae98721103119e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/8cac38229dad41aca20934d10c534a15, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/b600621526da40e983b98ff6049a2896, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/7bd67b069a404818a9fbc8e14a79273c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/30d2f03b14d14dfe8066c44377577432] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp, totalSize=26.6 K 2023-03-08 07:55:51,510 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting f84b2102c87d48279dae98721103119e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262142934 2023-03-08 07:55:51,511 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 8cac38229dad41aca20934d10c534a15, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262144711 2023-03-08 07:55:51,511 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b600621526da40e983b98ff6049a2896, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262146496 2023-03-08 07:55:51,512 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 7bd67b069a404818a9fbc8e14a79273c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262147882 2023-03-08 07:55:51,512 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 30d2f03b14d14dfe8066c44377577432, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:55:51,521 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): eba46d84043812856e232d435ccca635#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:51,928 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308ba6f2f7d37d048c5b23fb54bf5e89926_del 2023-03-08 07:55:51,928 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308ba6f2f7d37d048c5b23fb54bf5e89926_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308ba6f2f7d37d048c5b23fb54bf5e89926_del 2023-03-08 07:55:52,741 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f93a98763b3a4d7c819a21644e640735 2023-03-08 07:55:52,742 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family1/f93a98763b3a4d7c819a21644e640735 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f93a98763b3a4d7c819a21644e640735 2023-03-08 07:55:52,745 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f93a98763b3a4d7c819a21644e640735 2023-03-08 07:55:52,746 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in eba46d84043812856e232d435ccca635/family1 of eba46d84043812856e232d435ccca635 into f93a98763b3a4d7c819a21644e640735(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:52,746 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:52,746 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:52,746 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): eba46d84043812856e232d435ccca635/family2 is initiating major compaction (all files) 2023-03-08 07:55:52,746 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:52,746 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:52,746 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of eba46d84043812856e232d435ccca635/family2 in ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:55:52,746 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/aaebaf6e905541098ebc7b7104e4ee60, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/c451e5661c5a49619ca4b14a8e35c327, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/78d33d6ee2754ef0b4efa279c263120c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/af139451c42f4495b104381f2ac524f6, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/ffdaa47005514b1cb74ba1c1358345e9] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp, totalSize=25.3 K 2023-03-08 07:55:52,747 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting aaebaf6e905541098ebc7b7104e4ee60, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262142934 2023-03-08 07:55:52,747 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting c451e5661c5a49619ca4b14a8e35c327, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262144711 2023-03-08 07:55:52,747 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 78d33d6ee2754ef0b4efa279c263120c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262146496 2023-03-08 07:55:52,747 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting af139451c42f4495b104381f2ac524f6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262147882 2023-03-08 07:55:52,748 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting ffdaa47005514b1cb74ba1c1358345e9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:55:52,756 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): eba46d84043812856e232d435ccca635#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:52,837 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f84b2102c87d48279dae98721103119e, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/8cac38229dad41aca20934d10c534a15, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/b600621526da40e983b98ff6049a2896, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/7bd67b069a404818a9fbc8e14a79273c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/30d2f03b14d14dfe8066c44377577432] to archive 2023-03-08 07:55:52,838 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:52,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f84b2102c87d48279dae98721103119e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/f84b2102c87d48279dae98721103119e 2023-03-08 07:55:52,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/8cac38229dad41aca20934d10c534a15 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/8cac38229dad41aca20934d10c534a15 2023-03-08 07:55:52,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/b600621526da40e983b98ff6049a2896 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/b600621526da40e983b98ff6049a2896 2023-03-08 07:55:52,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/7bd67b069a404818a9fbc8e14a79273c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/7bd67b069a404818a9fbc8e14a79273c 2023-03-08 07:55:52,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/30d2f03b14d14dfe8066c44377577432 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/30d2f03b14d14dfe8066c44377577432 2023-03-08 07:55:53,164 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308e0b99aef84874914bf968fc0aa2560d8_del 2023-03-08 07:55:53,164 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308e0b99aef84874914bf968fc0aa2560d8_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308e0b99aef84874914bf968fc0aa2560d8_del 2023-03-08 07:55:53,978 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5aaa23f8a594c69b7840dc949952fde 2023-03-08 07:55:53,979 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/.tmp/family2/d5aaa23f8a594c69b7840dc949952fde as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/d5aaa23f8a594c69b7840dc949952fde 2023-03-08 07:55:53,982 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5aaa23f8a594c69b7840dc949952fde 2023-03-08 07:55:53,983 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in eba46d84043812856e232d435ccca635/family2 of eba46d84043812856e232d435ccca635 into d5aaa23f8a594c69b7840dc949952fde(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:53,983 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:53,983 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:53,983 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 05308de2ff490d772d5a663e6692c790/family1 is initiating major compaction (all files) 2023-03-08 07:55:53,983 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:53,983 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:53,984 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 05308de2ff490d772d5a663e6692c790/family1 in ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:53,984 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/82448cd237fe493a9d7832b3407957c1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/fc048389f3924791a003d6ea27f2108c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/25fb314d1fbd4e208992881777d246ec, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/40351e4efd75494984f47ce9149c6ec9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/1acdbd32e8c2449b9eda1d1f526721b5] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp, totalSize=26.6 K 2023-03-08 07:55:53,984 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 82448cd237fe493a9d7832b3407957c1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262142934 2023-03-08 07:55:53,984 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting fc048389f3924791a003d6ea27f2108c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262144711 2023-03-08 07:55:53,985 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 25fb314d1fbd4e208992881777d246ec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262146497 2023-03-08 07:55:53,985 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 40351e4efd75494984f47ce9149c6ec9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262147882 2023-03-08 07:55:53,985 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 1acdbd32e8c2449b9eda1d1f526721b5, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:55:53,992 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 05308de2ff490d772d5a663e6692c790#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:54,006 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030863e22e7980ee4885a47123571beb78e7_del 2023-03-08 07:55:54,006 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030863e22e7980ee4885a47123571beb78e7_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030863e22e7980ee4885a47123571beb78e7_del 2023-03-08 07:55:54,020 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83d44b1f0e064d97a4e5f37b07fc02a7 2023-03-08 07:55:54,020 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family1/83d44b1f0e064d97a4e5f37b07fc02a7 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/83d44b1f0e064d97a4e5f37b07fc02a7 2023-03-08 07:55:54,023 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83d44b1f0e064d97a4e5f37b07fc02a7 2023-03-08 07:55:54,024 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 05308de2ff490d772d5a663e6692c790/family1 of 05308de2ff490d772d5a663e6692c790 into 83d44b1f0e064d97a4e5f37b07fc02a7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:55:54,024 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:54,024 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:54,024 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 05308de2ff490d772d5a663e6692c790/family2 is initiating major compaction (all files) 2023-03-08 07:55:54,024 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:54,024 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:54,025 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 05308de2ff490d772d5a663e6692c790/family2 in ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:55:54,025 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/e80e0b25f8bd4ed98968f97da77e7665, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/34f688040c7e4dbcb6d0f54a2a9efe9f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/d5cbf731831b4ba282a423f86b509911, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/8e6f7a6b544f44dbb52e5317b1b5e359, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/589362f989e84566973123d297ef0e10] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp, totalSize=25.3 K 2023-03-08 07:55:54,025 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting e80e0b25f8bd4ed98968f97da77e7665, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262142934 2023-03-08 07:55:54,025 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 34f688040c7e4dbcb6d0f54a2a9efe9f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262144711 2023-03-08 07:55:54,025 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting d5cbf731831b4ba282a423f86b509911, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262146497 2023-03-08 07:55:54,026 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 8e6f7a6b544f44dbb52e5317b1b5e359, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262147882 2023-03-08 07:55:54,026 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 589362f989e84566973123d297ef0e10, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:55:54,033 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 05308de2ff490d772d5a663e6692c790#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:54,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/aaebaf6e905541098ebc7b7104e4ee60, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/c451e5661c5a49619ca4b14a8e35c327, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/78d33d6ee2754ef0b4efa279c263120c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/af139451c42f4495b104381f2ac524f6, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/ffdaa47005514b1cb74ba1c1358345e9] to archive 2023-03-08 07:55:54,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:54,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/aaebaf6e905541098ebc7b7104e4ee60 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/aaebaf6e905541098ebc7b7104e4ee60 2023-03-08 07:55:54,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/c451e5661c5a49619ca4b14a8e35c327 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/c451e5661c5a49619ca4b14a8e35c327 2023-03-08 07:55:54,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/78d33d6ee2754ef0b4efa279c263120c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/78d33d6ee2754ef0b4efa279c263120c 2023-03-08 07:55:54,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/af139451c42f4495b104381f2ac524f6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/af139451c42f4495b104381f2ac524f6 2023-03-08 07:55:54,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/ffdaa47005514b1cb74ba1c1358345e9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family2/ffdaa47005514b1cb74ba1c1358345e9 2023-03-08 07:55:54,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/82448cd237fe493a9d7832b3407957c1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/fc048389f3924791a003d6ea27f2108c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/25fb314d1fbd4e208992881777d246ec, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/40351e4efd75494984f47ce9149c6ec9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/1acdbd32e8c2449b9eda1d1f526721b5] to archive 2023-03-08 07:55:54,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:54,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/82448cd237fe493a9d7832b3407957c1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/82448cd237fe493a9d7832b3407957c1 2023-03-08 07:55:54,048 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/fc048389f3924791a003d6ea27f2108c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/fc048389f3924791a003d6ea27f2108c 2023-03-08 07:55:54,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/25fb314d1fbd4e208992881777d246ec to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/25fb314d1fbd4e208992881777d246ec 2023-03-08 07:55:54,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/40351e4efd75494984f47ce9149c6ec9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/40351e4efd75494984f47ce9149c6ec9 2023-03-08 07:55:54,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/1acdbd32e8c2449b9eda1d1f526721b5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/1acdbd32e8c2449b9eda1d1f526721b5 2023-03-08 07:55:54,443 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308b9a24d7fa8ea451e8471257267e8a171_del 2023-03-08 07:55:54,443 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b9a24d7fa8ea451e8471257267e8a171_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308b9a24d7fa8ea451e8471257267e8a171_del 2023-03-08 07:55:55,259 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 29b6c696727641b7a280de88b7a6725d 2023-03-08 07:55:55,260 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/.tmp/family2/29b6c696727641b7a280de88b7a6725d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/29b6c696727641b7a280de88b7a6725d 2023-03-08 07:55:55,263 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 29b6c696727641b7a280de88b7a6725d 2023-03-08 07:55:55,264 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 05308de2ff490d772d5a663e6692c790/family2 of 05308de2ff490d772d5a663e6692c790 into 29b6c696727641b7a280de88b7a6725d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:55,264 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:55,264 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:55,264 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): f3f6a8571aa8d2b03ba727aae5b43603/family1 is initiating major compaction (all files) 2023-03-08 07:55:55,264 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:55,264 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:55,265 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of f3f6a8571aa8d2b03ba727aae5b43603/family1 in ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:55,265 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/caddd693ceba4cf9b09c9d83337673b5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/1b30e136f26c4d629e6bf412e85334bd, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/0f8e90762d7141dca4fa616c7d6e8202, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/ec407ac0a74e4d108620b311fc3d85a0] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp, totalSize=26.6 K 2023-03-08 07:55:55,265 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting caddd693ceba4cf9b09c9d83337673b5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262142933 2023-03-08 07:55:55,266 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 1b30e136f26c4d629e6bf412e85334bd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262144711 2023-03-08 07:55:55,266 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting e6d2f1a1e02a446d92b22f07b6d4ca0c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262146496 2023-03-08 07:55:55,266 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 0f8e90762d7141dca4fa616c7d6e8202, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262147881 2023-03-08 07:55:55,267 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting ec407ac0a74e4d108620b311fc3d85a0, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:55:55,274 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): f3f6a8571aa8d2b03ba727aae5b43603#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:55,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/e80e0b25f8bd4ed98968f97da77e7665, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/34f688040c7e4dbcb6d0f54a2a9efe9f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/d5cbf731831b4ba282a423f86b509911, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/8e6f7a6b544f44dbb52e5317b1b5e359, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/589362f989e84566973123d297ef0e10] to archive 2023-03-08 07:55:55,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:55,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/e80e0b25f8bd4ed98968f97da77e7665 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/e80e0b25f8bd4ed98968f97da77e7665 2023-03-08 07:55:55,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/34f688040c7e4dbcb6d0f54a2a9efe9f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/34f688040c7e4dbcb6d0f54a2a9efe9f 2023-03-08 07:55:55,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/d5cbf731831b4ba282a423f86b509911 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/d5cbf731831b4ba282a423f86b509911 2023-03-08 07:55:55,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/8e6f7a6b544f44dbb52e5317b1b5e359 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/8e6f7a6b544f44dbb52e5317b1b5e359 2023-03-08 07:55:55,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/589362f989e84566973123d297ef0e10 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family2/589362f989e84566973123d297ef0e10 2023-03-08 07:55:55,682 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030810af153a7f8b4ba88205f8a20b09bbf3_del 2023-03-08 07:55:55,682 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030810af153a7f8b4ba88205f8a20b09bbf3_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030810af153a7f8b4ba88205f8a20b09bbf3_del 2023-03-08 07:55:56,496 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 969db9f9aac94a24a060bc2e947fe6d8 2023-03-08 07:55:56,497 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family1/969db9f9aac94a24a060bc2e947fe6d8 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/969db9f9aac94a24a060bc2e947fe6d8 2023-03-08 07:55:56,500 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 969db9f9aac94a24a060bc2e947fe6d8 2023-03-08 07:55:56,501 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f3f6a8571aa8d2b03ba727aae5b43603/family1 of f3f6a8571aa8d2b03ba727aae5b43603 into 969db9f9aac94a24a060bc2e947fe6d8(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:56,501 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:56,501 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:55:56,501 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): f3f6a8571aa8d2b03ba727aae5b43603/family2 is initiating major compaction (all files) 2023-03-08 07:55:56,501 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:55:56,501 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:55:56,501 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of f3f6a8571aa8d2b03ba727aae5b43603/family2 in ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:55:56,501 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/1dfacb090cde4b73b26aa687d5af0e77, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/0d7f9e02a7964211a2dc4ee9907a66c8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/f760838205134a809589c32e455a5008, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/2c7b33be55634d719fe510366a9187a1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/b1ffcf35ca734467a7bb415d2b116fd4] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp, totalSize=25.3 K 2023-03-08 07:55:56,502 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 1dfacb090cde4b73b26aa687d5af0e77, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262142933 2023-03-08 07:55:56,502 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 0d7f9e02a7964211a2dc4ee9907a66c8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262144711 2023-03-08 07:55:56,502 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting f760838205134a809589c32e455a5008, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262146496 2023-03-08 07:55:56,503 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 2c7b33be55634d719fe510366a9187a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262147881 2023-03-08 07:55:56,503 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b1ffcf35ca734467a7bb415d2b116fd4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:55:56,511 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): f3f6a8571aa8d2b03ba727aae5b43603#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:55:56,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/caddd693ceba4cf9b09c9d83337673b5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/1b30e136f26c4d629e6bf412e85334bd, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/0f8e90762d7141dca4fa616c7d6e8202, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/ec407ac0a74e4d108620b311fc3d85a0] to archive 2023-03-08 07:55:56,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:56,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/caddd693ceba4cf9b09c9d83337673b5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/caddd693ceba4cf9b09c9d83337673b5 2023-03-08 07:55:56,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/1b30e136f26c4d629e6bf412e85334bd to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/1b30e136f26c4d629e6bf412e85334bd 2023-03-08 07:55:56,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/e6d2f1a1e02a446d92b22f07b6d4ca0c 2023-03-08 07:55:56,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/0f8e90762d7141dca4fa616c7d6e8202 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/0f8e90762d7141dca4fa616c7d6e8202 2023-03-08 07:55:56,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/ec407ac0a74e4d108620b311fc3d85a0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/ec407ac0a74e4d108620b311fc3d85a0 2023-03-08 07:55:56,918 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308b62b71cfa50d4923a1263cfcb0c9581f_del 2023-03-08 07:55:56,918 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b62b71cfa50d4923a1263cfcb0c9581f_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308b62b71cfa50d4923a1263cfcb0c9581f_del 2023-03-08 07:55:57,731 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83aa88078b4b4df595653a99602b7271 2023-03-08 07:55:57,732 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/.tmp/family2/83aa88078b4b4df595653a99602b7271 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/83aa88078b4b4df595653a99602b7271 2023-03-08 07:55:57,735 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83aa88078b4b4df595653a99602b7271 2023-03-08 07:55:57,736 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f3f6a8571aa8d2b03ba727aae5b43603/family2 of f3f6a8571aa8d2b03ba727aae5b43603 into 83aa88078b4b4df595653a99602b7271(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:55:57,736 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:57,739 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:57,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/1dfacb090cde4b73b26aa687d5af0e77, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/0d7f9e02a7964211a2dc4ee9907a66c8, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/f760838205134a809589c32e455a5008, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/2c7b33be55634d719fe510366a9187a1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/b1ffcf35ca734467a7bb415d2b116fd4] to archive 2023-03-08 07:55:57,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:57,742 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:57,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/1dfacb090cde4b73b26aa687d5af0e77 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/1dfacb090cde4b73b26aa687d5af0e77 2023-03-08 07:55:57,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/0d7f9e02a7964211a2dc4ee9907a66c8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/0d7f9e02a7964211a2dc4ee9907a66c8 2023-03-08 07:55:57,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/f760838205134a809589c32e455a5008 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/f760838205134a809589c32e455a5008 2023-03-08 07:55:57,746 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:57,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/2c7b33be55634d719fe510366a9187a1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/2c7b33be55634d719fe510366a9187a1 2023-03-08 07:55:57,747 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:57,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/b1ffcf35ca734467a7bb415d2b116fd4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family2/b1ffcf35ca734467a7bb415d2b116fd4 2023-03-08 07:55:57,753 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 07:55:57,766 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-08 07:55:57,768 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030810af153a7f8b4ba88205f8a20b09bbf3_del 2023-03-08 07:55:57,771 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308ba6f2f7d37d048c5b23fb54bf5e89926_del 2023-03-08 07:55:57,773 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030863e22e7980ee4885a47123571beb78e7_del 2023-03-08 07:55:57,773 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:57,773 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(139): Connect 0x23871a53 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:55:57,777 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4022ea69, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:55:57,778 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 07:55:57,779 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 07:55:57,779 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 07:55:58,605 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303082474b0caba2642e28f1f9b7008df0f26 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303082474b0caba2642e28f1f9b7008df0f26 2023-03-08 07:55:58,606 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,609 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308fb37cf755a0c4f16bdb02d5154255389 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308fb37cf755a0c4f16bdb02d5154255389 2023-03-08 07:55:58,610 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,613 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303085b29f16905b047e0b83838c66f66859e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303085b29f16905b047e0b83838c66f66859e 2023-03-08 07:55:58,615 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,619 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:55:58,622 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43460, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:55:58,634 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:55:58,637 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42236, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:55:58,658 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/d914caac600c4b91b6cbe11b62800718 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-08 07:55:58,658 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/404031cc31854a76a8cb01a156b98eed first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-08 07:55:58,658 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9bb846e9b500485e85843982fd3adbed first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-08 07:55:58,659 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/404031cc31854a76a8cb01a156b98eed}] 2023-03-08 07:55:58,659 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/d914caac600c4b91b6cbe11b62800718}] 2023-03-08 07:55:58,660 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9bb846e9b500485e85843982fd3adbed}] 2023-03-08 07:55:58,667 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9bb846e9b500485e85843982fd3adbed for inclusion in f3f6a8571aa8d2b03ba727aae5b43603/family1 2023-03-08 07:55:58,667 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/404031cc31854a76a8cb01a156b98eed for inclusion in eba46d84043812856e232d435ccca635/family1 2023-03-08 07:55:58,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/d914caac600c4b91b6cbe11b62800718 for inclusion in 05308de2ff490d772d5a663e6692c790/family1 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 07:55:58,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:55:58,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:55:58,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/404031cc31854a76a8cb01a156b98eed to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__8ld32ov8nkbe5cvu6la8nmgl37ga7a84026461tp9tjm088jin5u7mf66jq7j1ha/family1/404031cc31854a76a8cb01a156b98eed 2023-03-08 07:55:58,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/d914caac600c4b91b6cbe11b62800718 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__jnkgpjrfl3pkrko7h6jj5lrokoo81upps0p0pjm0jnu1lu2kpraqqlj3rtq2fkp7/family1/d914caac600c4b91b6cbe11b62800718 2023-03-08 07:55:58,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/9bb846e9b500485e85843982fd3adbed to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__geh170f3josijv22d5bfi21lirk7qs0sigs6jgun3f182kdc0unfqd8qec9co0ps/family1/9bb846e9b500485e85843982fd3adbed 2023-03-08 07:55:58,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__jnkgpjrfl3pkrko7h6jj5lrokoo81upps0p0pjm0jnu1lu2kpraqqlj3rtq2fkp7/family1/d914caac600c4b91b6cbe11b62800718 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/e5cde551c6904211b3be2c79d1f67b91_SeqId_28_ 2023-03-08 07:55:58,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__8ld32ov8nkbe5cvu6la8nmgl37ga7a84026461tp9tjm088jin5u7mf66jq7j1ha/family1/404031cc31854a76a8cb01a156b98eed as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/5d60b724e0ba4a048b1d580633cbab57_SeqId_28_ 2023-03-08 07:55:58,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__jnkgpjrfl3pkrko7h6jj5lrokoo81upps0p0pjm0jnu1lu2kpraqqlj3rtq2fkp7/family1/d914caac600c4b91b6cbe11b62800718 into 05308de2ff490d772d5a663e6692c790/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/e5cde551c6904211b3be2c79d1f67b91_SeqId_28_ - updating store file list. 2023-03-08 07:55:58,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__geh170f3josijv22d5bfi21lirk7qs0sigs6jgun3f182kdc0unfqd8qec9co0ps/family1/9bb846e9b500485e85843982fd3adbed as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/d6dad3b1f00649389508a0ac1f6231b7_SeqId_28_ 2023-03-08 07:55:58,674 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__8ld32ov8nkbe5cvu6la8nmgl37ga7a84026461tp9tjm088jin5u7mf66jq7j1ha/family1/404031cc31854a76a8cb01a156b98eed into eba46d84043812856e232d435ccca635/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/5d60b724e0ba4a048b1d580633cbab57_SeqId_28_ - updating store file list. 2023-03-08 07:55:58,674 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__geh170f3josijv22d5bfi21lirk7qs0sigs6jgun3f182kdc0unfqd8qec9co0ps/family1/9bb846e9b500485e85843982fd3adbed into f3f6a8571aa8d2b03ba727aae5b43603/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/d6dad3b1f00649389508a0ac1f6231b7_SeqId_28_ - updating store file list. 2023-03-08 07:55:58,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/e5cde551c6904211b3be2c79d1f67b91_SeqId_28_ into 05308de2ff490d772d5a663e6692c790/family1 2023-03-08 07:55:58,678 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__jnkgpjrfl3pkrko7h6jj5lrokoo81upps0p0pjm0jnu1lu2kpraqqlj3rtq2fkp7/family1/d914caac600c4b91b6cbe11b62800718 into 05308de2ff490d772d5a663e6692c790/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/family1/e5cde551c6904211b3be2c79d1f67b91_SeqId_28_) 2023-03-08 07:55:58,678 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/5d60b724e0ba4a048b1d580633cbab57_SeqId_28_ into eba46d84043812856e232d435ccca635/family1 2023-03-08 07:55:58,678 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__8ld32ov8nkbe5cvu6la8nmgl37ga7a84026461tp9tjm088jin5u7mf66jq7j1ha/family1/404031cc31854a76a8cb01a156b98eed into eba46d84043812856e232d435ccca635/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/family1/5d60b724e0ba4a048b1d580633cbab57_SeqId_28_) 2023-03-08 07:55:58,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/d6dad3b1f00649389508a0ac1f6231b7_SeqId_28_ into f3f6a8571aa8d2b03ba727aae5b43603/family1 2023-03-08 07:55:58,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__geh170f3josijv22d5bfi21lirk7qs0sigs6jgun3f182kdc0unfqd8qec9co0ps/family1/9bb846e9b500485e85843982fd3adbed into f3f6a8571aa8d2b03ba727aae5b43603/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/family1/d6dad3b1f00649389508a0ac1f6231b7_SeqId_28_) 2023-03-08 07:55:58,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__jnkgpjrfl3pkrko7h6jj5lrokoo81upps0p0pjm0jnu1lu2kpraqqlj3rtq2fkp7/family1/d914caac600c4b91b6cbe11b62800718 2023-03-08 07:55:58,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__8ld32ov8nkbe5cvu6la8nmgl37ga7a84026461tp9tjm088jin5u7mf66jq7j1ha/family1/404031cc31854a76a8cb01a156b98eed 2023-03-08 07:55:58,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__ns_testMinorCompaction__geh170f3josijv22d5bfi21lirk7qs0sigs6jgun3f182kdc0unfqd8qec9co0ps/family1/9bb846e9b500485e85843982fd3adbed 2023-03-08 07:55:58,682 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,682 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,682 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,683 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:58,683 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:58,683 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:55:58,684 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030877346283e2b34d49a334294b9ea98d76 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030877346283e2b34d49a334294b9ea98d76 2023-03-08 07:55:58,684 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 07:55:58,684 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 07:55:58,684 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ed0fce9b081047eab78288255f1076aa to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ed0fce9b081047eab78288255f1076aa 2023-03-08 07:55:58,684 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303088a7c9708a2bd40a694b83f827af975a8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303088a7c9708a2bd40a694b83f827af975a8 2023-03-08 07:55:58,684 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 07:55:58,684 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 07:55:58,684 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 07:55:58,684 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 07:55:58,685 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 07:55:58,685 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 07:55:58,685 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 07:55:58,685 INFO [Listener at localhost/42999] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-08 07:55:58,687 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:58,689 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:58,691 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:55:58,692 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:55:58,710 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=248 (was 290), OpenFileDescriptor=622 (was 768), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=168 (was 187), ProcessCount=170 (was 170), AvailableMemoryMB=7442 (was 7452) 2023-03-08 07:55:58,722 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=168, ProcessCount=170, AvailableMemoryMB=7441 2023-03-08 07:55:58,724 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 07:55:58,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-08 07:55:58,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:55:58,727 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-08 07:55:58,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:55:58,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 07:55:58,728 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:58,729 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1 empty. 2023-03-08 07:55:58,729 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:58,729 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-08 07:55:58,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 07:55:59,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 07:55:59,136 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 07:55:59,137 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8edf959f1d606a2cc1723c3487e05ae1, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:55:59,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 07:55:59,542 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:59,542 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 8edf959f1d606a2cc1723c3487e05ae1, disabling compactions & flushes 2023-03-08 07:55:59,542 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,542 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,542 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. after waiting 0 ms 2023-03-08 07:55:59,542 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,542 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,542 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:55:59,544 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:55:59,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678262159544"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262159544"}]},"ts":"1678262159544"} 2023-03-08 07:55:59,546 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 07:55:59,547 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:55:59,547 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262159547"}]},"ts":"1678262159547"} 2023-03-08 07:55:59,548 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 07:55:59,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=8edf959f1d606a2cc1723c3487e05ae1, ASSIGN}] 2023-03-08 07:55:59,553 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=8edf959f1d606a2cc1723c3487e05ae1, ASSIGN 2023-03-08 07:55:59,553 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=8edf959f1d606a2cc1723c3487e05ae1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:55:59,704 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=8edf959f1d606a2cc1723c3487e05ae1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:59,705 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678262159704"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262159704"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262159704"}]},"ts":"1678262159704"} 2023-03-08 07:55:59,706 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 8edf959f1d606a2cc1723c3487e05ae1, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:55:59,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 07:55:59,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8edf959f1d606a2cc1723c3487e05ae1, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 07:55:59,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:55:59,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,863 INFO [StoreOpener-8edf959f1d606a2cc1723c3487e05ae1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,864 DEBUG [StoreOpener-8edf959f1d606a2cc1723c3487e05ae1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:55:59,864 DEBUG [StoreOpener-8edf959f1d606a2cc1723c3487e05ae1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:55:59,864 INFO [StoreOpener-8edf959f1d606a2cc1723c3487e05ae1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8edf959f1d606a2cc1723c3487e05ae1 columnFamilyName family1 2023-03-08 07:55:59,865 INFO [StoreOpener-8edf959f1d606a2cc1723c3487e05ae1-1] regionserver.HStore(310): Store=8edf959f1d606a2cc1723c3487e05ae1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:55:59,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:55:59,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:55:59,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8edf959f1d606a2cc1723c3487e05ae1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61952189, jitterRate=-0.07684044539928436}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 07:55:59,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:55:59,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1., pid=114, masterSystemTime=1678262159857 2023-03-08 07:55:59,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:55:59,873 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=8edf959f1d606a2cc1723c3487e05ae1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:55:59,873 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678262159873"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262159873"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262159873"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262159873"}]},"ts":"1678262159873"} 2023-03-08 07:55:59,875 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-08 07:55:59,875 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 8edf959f1d606a2cc1723c3487e05ae1, server=jenkins-hbase4.apache.org,33669,1678262016084 in 168 msec 2023-03-08 07:55:59,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-08 07:55:59,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=8edf959f1d606a2cc1723c3487e05ae1, ASSIGN in 323 msec 2023-03-08 07:55:59,877 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:55:59,877 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262159877"}]},"ts":"1678262159877"} 2023-03-08 07:55:59,878 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 07:55:59,879 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:55:59,880 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1550 sec 2023-03-08 07:56:00,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 07:56:00,832 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-08 07:56:00,853 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:00,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,860 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:00,860 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:00,860 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:00,860 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:00,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,862 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:00,862 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:00,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:00,863 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,863 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:00,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:00,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,864 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:00,864 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:00,864 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:00,865 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:00,865 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:00,865 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:00,865 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:00,866 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 07:56:00,889 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511303d6d263ee25346079088d300ad377eb9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511303d6d263ee25346079088d300ad377eb9 2023-03-08 07:56:00,893 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/8778ca589d36424f8ce78b7a89f90e32 2023-03-08 07:56:00,897 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/8778ca589d36424f8ce78b7a89f90e32 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/8778ca589d36424f8ce78b7a89f90e32 2023-03-08 07:56:00,900 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/8778ca589d36424f8ce78b7a89f90e32, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 07:56:00,901 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 35ms, sequenceid=5, compaction requested=false 2023-03-08 07:56:00,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:00,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:00,901 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:00,901 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:00,902 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:00,902 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,902 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:00,902 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:00,903 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,903 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:00,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:00,903 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,903 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:00,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:00,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:00,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:00,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:00,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@289d1d19[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:00,905 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:00,905 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,907 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:00,907 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,907 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:00,909 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,909 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:00,909 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:00,909 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:00,911 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:00,911 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:00,911 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:00,911 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:00,911 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,911 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:00,912 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,912 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,912 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:00,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:00,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,913 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:00,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,914 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,916 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:00,916 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:00,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:56:00,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:00,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:00,916 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:00,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:00,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:00,916 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:01,017 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:01,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:01,022 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:01,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:01,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,023 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:01,023 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:01,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:01,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:01,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,024 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,026 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,027 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:01,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:01,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:01,027 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,027 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:01,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:01,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,028 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:01,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:01,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:01,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:01,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:01,028 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:01,028 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:01,028 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:01,028 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 07:56:01,439 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128be3c49ef85f94c61902b9e580b0c4f47 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128be3c49ef85f94c61902b9e580b0c4f47 2023-03-08 07:56:01,845 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/847124a51d5e4c77986fcbffad4e8da5 2023-03-08 07:56:01,849 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/847124a51d5e4c77986fcbffad4e8da5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/847124a51d5e4c77986fcbffad4e8da5 2023-03-08 07:56:01,852 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/847124a51d5e4c77986fcbffad4e8da5, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 07:56:01,853 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 825ms, sequenceid=9, compaction requested=false 2023-03-08 07:56:01,853 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:01,853 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:01,853 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:01,853 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:01,853 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,853 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:01,853 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:01,855 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,855 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:01,855 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,855 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:01,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:01,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f831978[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:01,856 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:01,856 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,857 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,858 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:01,858 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:01,858 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:01,858 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,858 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:01,859 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,859 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:01,859 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:01,859 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:01,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:01,861 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:01,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:01,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:01,861 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,861 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:01,863 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,863 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:01,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,863 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:01,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:01,864 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:01,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:01,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:01,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,865 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:01,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:01,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:56:01,867 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:01,867 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,867 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:01,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:01,867 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:01,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:01,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:01,967 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:01,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:01,971 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:01,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:01,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,973 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:01,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:01,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:01,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:01,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,973 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,978 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:01,978 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:01,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:01,978 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:01,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,978 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:01,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:01,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:01,979 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:01,979 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:01,979 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:02,391 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201668d91912c89472cb578cea01e80780d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201668d91912c89472cb578cea01e80780d 2023-03-08 07:56:02,396 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/b417731b8f544282ac91685d02e500a4 2023-03-08 07:56:02,399 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/b417731b8f544282ac91685d02e500a4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/b417731b8f544282ac91685d02e500a4 2023-03-08 07:56:02,402 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/b417731b8f544282ac91685d02e500a4, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 07:56:02,402 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 423ms, sequenceid=13, compaction requested=true 2023-03-08 07:56:02,403 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:02,403 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:02,403 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:02,403 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:02,403 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,403 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:02,403 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:02,405 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,405 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:02,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:02,405 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,405 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:02,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20b4f5ec[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:02,406 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:02,407 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,407 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,408 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:02,408 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:02,408 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:02,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:02,409 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:02,409 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:02,409 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,409 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:02,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:02,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:02,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:02,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:02,411 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:02,411 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:02,411 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:02,411 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,411 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:02,412 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,412 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:02,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,413 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:02,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:02,413 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,414 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,417 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,417 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,417 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:02,417 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:02,417 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:02,417 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:02,418 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:02,417 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:02,417 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:02,418 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,418 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,418 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,418 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:02,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:02,418 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:02,518 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:02,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:02,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:02,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:02,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,524 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:02,524 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:02,524 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:02,524 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:02,525 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,525 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,526 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,526 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:02,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:02,526 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:02,526 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:02,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,527 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,527 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:02,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:02,527 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:02,527 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:02,527 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:02,527 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:02,527 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:02,527 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:02,527 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:02,527 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:02,938 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053e29c6c8036b4c799d05e904f90d5994 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512053e29c6c8036b4c799d05e904f90d5994 2023-03-08 07:56:03,343 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/c30d53b730444c1abfecdc7f8873f432 2023-03-08 07:56:03,347 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/c30d53b730444c1abfecdc7f8873f432 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c30d53b730444c1abfecdc7f8873f432 2023-03-08 07:56:03,350 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c30d53b730444c1abfecdc7f8873f432, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 07:56:03,351 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 824ms, sequenceid=17, compaction requested=true 2023-03-08 07:56:03,351 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:03,351 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:03,351 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:03,351 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:03,351 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,351 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:03,351 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:03,353 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,353 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:03,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:03,353 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,353 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:03,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:03,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:03,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:03,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:03,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@266d0117[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:03,354 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:03,355 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,356 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,356 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:03,356 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:03,356 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,356 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:03,356 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:03,357 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:03,357 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,357 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:03,357 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:03,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:03,359 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:03,359 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:03,359 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:03,359 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,359 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:03,361 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,362 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:03,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,362 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:03,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:03,362 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:03,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,363 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:03,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,365 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,365 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:03,365 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:03,366 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:03,365 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,365 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:03,366 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,366 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:03,366 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:03,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:03,366 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,366 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,366 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:56:03,366 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:03,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:03,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:03,366 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:03,466 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:03,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:03,471 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:03,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:03,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,473 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:03,473 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:03,473 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:03,473 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:03,473 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,473 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,474 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,474 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:03,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:03,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:03,475 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:03,475 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:03,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:03,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:03,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:03,476 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:03,476 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:03,476 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:03,476 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:03,476 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:03,476 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:03,476 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:03,476 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:03,887 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bcaaee0181064f998a6860c1b4f0a9a7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228bcaaee0181064f998a6860c1b4f0a9a7 2023-03-08 07:56:04,292 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/1af8259947ae444ba09d5160c33cbf85 2023-03-08 07:56:04,295 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/1af8259947ae444ba09d5160c33cbf85 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/1af8259947ae444ba09d5160c33cbf85 2023-03-08 07:56:04,298 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/1af8259947ae444ba09d5160c33cbf85, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 07:56:04,299 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 823ms, sequenceid=21, compaction requested=true 2023-03-08 07:56:04,299 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:04,299 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:04,301 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,301 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:04,301 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,301 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:04,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:04,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6658b516[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:04,302 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:04,302 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,304 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:04,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:04,304 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:04,304 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,304 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:04,306 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:04,306 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,306 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:04,306 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:04,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:04,308 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:04,308 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:04,308 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:04,308 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,308 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:04,309 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,309 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:04,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,309 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:04,309 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,310 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:04,315 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:04,315 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,315 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:04,315 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,316 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,316 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:04,316 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,316 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:04,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:04,416 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:04,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:04,420 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:04,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:04,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,422 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:04,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:04,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:04,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:04,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,422 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,423 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,423 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:04,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:04,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:04,424 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:04,424 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:04,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:04,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,424 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:04,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:04,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:04,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:04,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:04,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:04,425 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:04,425 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:04,425 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:04,425 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:04,835 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316ea9cce38c6b4ce590ebc64061edb782 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316ea9cce38c6b4ce590ebc64061edb782 2023-03-08 07:56:05,240 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/072b99c31be5491a9f1822f42983e83f 2023-03-08 07:56:05,246 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/072b99c31be5491a9f1822f42983e83f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/072b99c31be5491a9f1822f42983e83f 2023-03-08 07:56:05,249 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/072b99c31be5491a9f1822f42983e83f, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 07:56:05,249 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 824ms, sequenceid=25, compaction requested=true 2023-03-08 07:56:05,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:05,249 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:05,249 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:05,249 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:05,250 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,250 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:05,250 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:05,251 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:05,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:05,252 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:05,252 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:05,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:05,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7107387c[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:05,253 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:05,253 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,255 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:05,255 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:05,255 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,256 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:05,257 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:05,257 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:05,257 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,257 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:05,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:05,259 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:05,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:05,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:05,259 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,259 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:05,260 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,260 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:05,260 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,261 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:05,262 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,263 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,263 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:05,264 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:05,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:56:05,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:05,263 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:05,264 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,264 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,264 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:05,264 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:05,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:05,364 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:05,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:05,369 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:05,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:05,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,370 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:05,370 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:05,370 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:05,370 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:05,371 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,371 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,373 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,373 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:05,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:05,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:05,374 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:05,374 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:05,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:05,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:05,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:05,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:05,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:05,375 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:05,375 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:05,375 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:05,375 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:05,398 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:56:05,786 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010ba3cc4462fd46f5ae09d79e7145ac3a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601010ba3cc4462fd46f5ae09d79e7145ac3a 2023-03-08 07:56:06,191 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/dcc8cce47cd64a158e43066dd296a3e2 2023-03-08 07:56:06,195 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/dcc8cce47cd64a158e43066dd296a3e2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/dcc8cce47cd64a158e43066dd296a3e2 2023-03-08 07:56:06,197 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/dcc8cce47cd64a158e43066dd296a3e2, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 07:56:06,198 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 823ms, sequenceid=29, compaction requested=true 2023-03-08 07:56:06,198 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:06,198 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:06,198 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:06,198 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:06,198 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:06,199 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:06,200 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,200 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:06,200 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,200 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:06,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:06,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@761ba825[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:06,202 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:06,202 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,203 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:06,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:06,203 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,203 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:06,203 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:06,204 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:06,204 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,204 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:06,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:06,204 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:06,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:06,206 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:06,206 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:06,206 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:06,206 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,207 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:06,209 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,209 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,209 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:06,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:06,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:06,210 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:06,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:06,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:06,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:06,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,211 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:06,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,212 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,212 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:06,212 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:06,213 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:06,212 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:06,213 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:06,213 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:06,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:06,212 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,213 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,213 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,213 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:56:06,213 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:06,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:06,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:06,213 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,213 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:06,313 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:06,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:06,318 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:06,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:06,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,319 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:06,319 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:06,319 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:06,319 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:06,320 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,320 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,321 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:06,321 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:06,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:06,321 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:06,321 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:06,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:06,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:06,323 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:06,323 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:06,323 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:06,323 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:06,323 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:06,323 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:06,323 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:06,323 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:06,323 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:06,734 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010350041959e8c74326a9b6139a316c9ee7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010350041959e8c74326a9b6139a316c9ee7 2023-03-08 07:56:07,145 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/cc5ab4cebd984e18a19c6464637866b2 2023-03-08 07:56:07,148 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/cc5ab4cebd984e18a19c6464637866b2 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/cc5ab4cebd984e18a19c6464637866b2 2023-03-08 07:56:07,151 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/cc5ab4cebd984e18a19c6464637866b2, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 07:56:07,152 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 829ms, sequenceid=33, compaction requested=true 2023-03-08 07:56:07,152 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:07,152 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:07,152 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:07,152 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:07,153 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,153 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:07,153 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:07,154 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,154 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:07,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,155 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:07,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:07,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38b06a48[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:07,156 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:07,156 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,158 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,158 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:07,158 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:07,158 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:07,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,158 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:07,159 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:07,159 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:07,159 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,159 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:07,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:07,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:07,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:07,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:07,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:07,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:07,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:07,161 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:07,161 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:07,161 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:07,161 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,161 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:07,163 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,163 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:07,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:07,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,163 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:07,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:07,163 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:07,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:07,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:07,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,164 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:07,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,170 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,170 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:07,170 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,171 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:07,171 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,171 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:07,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:07,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:07,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:07,171 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,271 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:07,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:07,275 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:07,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:07,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,277 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:07,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:07,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:07,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:07,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,277 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,279 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:07,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:07,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:07,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:07,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:07,280 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:07,280 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:07,280 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:07,280 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:07,280 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:07,280 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:07,280 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:07,280 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:07,691 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907056202b39e364584970a8e2e246b94de to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907056202b39e364584970a8e2e246b94de 2023-03-08 07:56:08,096 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/e9eb9a6547a8450d9674b4894e9d115d 2023-03-08 07:56:08,100 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/e9eb9a6547a8450d9674b4894e9d115d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/e9eb9a6547a8450d9674b4894e9d115d 2023-03-08 07:56:08,102 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/e9eb9a6547a8450d9674b4894e9d115d, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 07:56:08,103 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 823ms, sequenceid=37, compaction requested=true 2023-03-08 07:56:08,103 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:08,103 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:08,103 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:08,103 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:08,103 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,103 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:08,103 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:08,105 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,105 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:08,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:08,105 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,105 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:08,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:08,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:08,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b63984d[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:08,107 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:08,107 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,108 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,108 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:08,108 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:08,108 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:08,108 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,108 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:08,110 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,110 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:08,110 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:08,110 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:08,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:08,112 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:08,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:08,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:08,112 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,112 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:08,115 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,115 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,115 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:08,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:08,115 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:08,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:08,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,116 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:08,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:08,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:08,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 07:56:08,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:08,118 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:08,119 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:08,118 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:08,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,118 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,119 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,219 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:08,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:08,223 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:08,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:08,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,225 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:08,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:08,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 07:56:08,225 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 07:56:08,226 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,226 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,227 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,227 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:08,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:08,228 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:08,228 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 07:56:08,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:08,229 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 07:56:08,229 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:08,229 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 07:56:08,229 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 07:56:08,229 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:08,229 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:08,229 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. started... 2023-03-08 07:56:08,229 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8edf959f1d606a2cc1723c3487e05ae1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 07:56:08,640 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a6bc0b077a72429e9bc001ecaf0b2160 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120a6bc0b077a72429e9bc001ecaf0b2160 2023-03-08 07:56:09,045 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/2a297909a9574b8c97d3f05a803619fa 2023-03-08 07:56:09,049 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/2a297909a9574b8c97d3f05a803619fa as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/2a297909a9574b8c97d3f05a803619fa 2023-03-08 07:56:09,052 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/2a297909a9574b8c97d3f05a803619fa, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 07:56:09,052 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8edf959f1d606a2cc1723c3487e05ae1 in 823ms, sequenceid=41, compaction requested=true 2023-03-08 07:56:09,053 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:09,053 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 07:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 07:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 07:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 07:56:09,054 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:09,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 07:56:09,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@536f8785[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:09,056 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 07:56:09,056 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,058 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,058 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 07:56:09,058 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 07:56:09,058 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,058 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:09,058 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:09,059 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 07:56:09,059 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,059 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:09,060 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 07:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:09,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:09,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 07:56:09,061 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 07:56:09,062 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:09,062 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:09,062 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,062 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:09,063 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,063 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:09,063 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:09,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,064 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:09,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,065 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,067 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,067 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:09,067 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:09,068 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:09,067 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:09,168 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:09,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:09,170 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-08 07:56:09,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:56:09,172 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:56:09,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:56:09,172 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262769172 2023-03-08 07:56:09,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 07:56:09,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 07:56:09,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:09,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x6b4be601 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:56:09,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@112ab7c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:56:09,180 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:10,002 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033aaf3b91d6a64ada84d246f24acc659f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601033aaf3b91d6a64ada84d246f24acc659f 2023-03-08 07:56:10,003 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:10,015 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:56:10,017 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52682, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:56:10,023 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:56:10,024 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52742, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:56:10,031 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/5bbbb1a4276f4161ab9e3da77fa49444 first=Optional[r5] last=Optional[r6] 2023-03-08 07:56:10,031 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/5bbbb1a4276f4161ab9e3da77fa49444}] 2023-03-08 07:56:10,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/5bbbb1a4276f4161ab9e3da77fa49444 for inclusion in 8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:56:10,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 07:56:10,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:56:10,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:10,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/5bbbb1a4276f4161ab9e3da77fa49444 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rmi5m9p4ju2q3l8phf4teickve177eeho2j050vqgr5k1ci4drrvuc9n5lm9eg2h/family1/5bbbb1a4276f4161ab9e3da77fa49444 2023-03-08 07:56:10,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rmi5m9p4ju2q3l8phf4teickve177eeho2j050vqgr5k1ci4drrvuc9n5lm9eg2h/family1/5bbbb1a4276f4161ab9e3da77fa49444 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_ 2023-03-08 07:56:10,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rmi5m9p4ju2q3l8phf4teickve177eeho2j050vqgr5k1ci4drrvuc9n5lm9eg2h/family1/5bbbb1a4276f4161ab9e3da77fa49444 into 8edf959f1d606a2cc1723c3487e05ae1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_ - updating store file list. 2023-03-08 07:56:10,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_ into 8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:56:10,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rmi5m9p4ju2q3l8phf4teickve177eeho2j050vqgr5k1ci4drrvuc9n5lm9eg2h/family1/5bbbb1a4276f4161ab9e3da77fa49444 into 8edf959f1d606a2cc1723c3487e05ae1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_) 2023-03-08 07:56:10,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rmi5m9p4ju2q3l8phf4teickve177eeho2j050vqgr5k1ci4drrvuc9n5lm9eg2h/family1/5bbbb1a4276f4161ab9e3da77fa49444 2023-03-08 07:56:10,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 07:56:10,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 07:56:10,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 8edf959f1d606a2cc1723c3487e05ae1/family1 is initiating minor compaction 2023-03-08 07:56:10,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@77ba0590; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:56:10,048 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8edf959f1d606a2cc1723c3487e05ae1/family1 in testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:10,048 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/847124a51d5e4c77986fcbffad4e8da5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/b417731b8f544282ac91685d02e500a4, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c30d53b730444c1abfecdc7f8873f432, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/1af8259947ae444ba09d5160c33cbf85, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/072b99c31be5491a9f1822f42983e83f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/dcc8cce47cd64a158e43066dd296a3e2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/cc5ab4cebd984e18a19c6464637866b2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/e9eb9a6547a8450d9674b4894e9d115d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/2a297909a9574b8c97d3f05a803619fa, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp, totalSize=49.7 K 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 847124a51d5e4c77986fcbffad4e8da5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting b417731b8f544282ac91685d02e500a4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting c30d53b730444c1abfecdc7f8873f432, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 1af8259947ae444ba09d5160c33cbf85, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 072b99c31be5491a9f1822f42983e83f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting dcc8cce47cd64a158e43066dd296a3e2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting cc5ab4cebd984e18a19c6464637866b2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting e9eb9a6547a8450d9674b4894e9d115d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 2a297909a9574b8c97d3f05a803619fa, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 07:56:10,049 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 07:56:10,050 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:10,051 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:10,052 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601010ba3cc4462fd46f5ae09d79e7145ac3a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601010ba3cc4462fd46f5ae09d79e7145ac3a 2023-03-08 07:56:10,053 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010350041959e8c74326a9b6139a316c9ee7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010350041959e8c74326a9b6139a316c9ee7 2023-03-08 07:56:10,053 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 07:56:10,053 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 07:56:10,054 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:10,054 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:10,084 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8edf959f1d606a2cc1723c3487e05ae1#family1#compaction#165 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:10,099 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123152e38a7a3a6f455ab3e883f8022645d3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123152e38a7a3a6f455ab3e883f8022645d3 2023-03-08 07:56:10,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:10,121 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/925fbb92d5254660a6f38763e6e8dd1e first=Optional[r1] last=Optional[r4] 2023-03-08 07:56:10,122 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/925fbb92d5254660a6f38763e6e8dd1e}] 2023-03-08 07:56:10,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/925fbb92d5254660a6f38763e6e8dd1e for inclusion in 8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:56:10,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 07:56:10,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:56:10,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:10,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/925fbb92d5254660a6f38763e6e8dd1e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oe4rhkhn3k72e0v4lkhtjmbqj8jm423h7kacc1fc8gbkgcvahtbpu1tpebdj5t50/family1/925fbb92d5254660a6f38763e6e8dd1e 2023-03-08 07:56:10,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oe4rhkhn3k72e0v4lkhtjmbqj8jm423h7kacc1fc8gbkgcvahtbpu1tpebdj5t50/family1/925fbb92d5254660a6f38763e6e8dd1e as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_ 2023-03-08 07:56:10,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oe4rhkhn3k72e0v4lkhtjmbqj8jm423h7kacc1fc8gbkgcvahtbpu1tpebdj5t50/family1/925fbb92d5254660a6f38763e6e8dd1e into 8edf959f1d606a2cc1723c3487e05ae1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_ - updating store file list. 2023-03-08 07:56:10,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_ into 8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:56:10,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oe4rhkhn3k72e0v4lkhtjmbqj8jm423h7kacc1fc8gbkgcvahtbpu1tpebdj5t50/family1/925fbb92d5254660a6f38763e6e8dd1e into 8edf959f1d606a2cc1723c3487e05ae1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_) 2023-03-08 07:56:10,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oe4rhkhn3k72e0v4lkhtjmbqj8jm423h7kacc1fc8gbkgcvahtbpu1tpebdj5t50/family1/925fbb92d5254660a6f38763e6e8dd1e 2023-03-08 07:56:10,132 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:10,133 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:10,134 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201668d91912c89472cb578cea01e80780d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201668d91912c89472cb578cea01e80780d 2023-03-08 07:56:10,135 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512053e29c6c8036b4c799d05e904f90d5994 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512053e29c6c8036b4c799d05e904f90d5994 2023-03-08 07:56:10,136 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228bcaaee0181064f998a6860c1b4f0a9a7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228bcaaee0181064f998a6860c1b4f0a9a7 2023-03-08 07:56:10,136 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316ea9cce38c6b4ce590ebc64061edb782 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316ea9cce38c6b4ce590ebc64061edb782 2023-03-08 07:56:10,137 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 07:56:10,137 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 07:56:10,137 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:10,137 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:10,897 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/ecaeef0337104dd1875ddb991ad64d57 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ecaeef0337104dd1875ddb991ad64d57 2023-03-08 07:56:10,902 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8edf959f1d606a2cc1723c3487e05ae1/family1 of 8edf959f1d606a2cc1723c3487e05ae1 into ecaeef0337104dd1875ddb991ad64d57(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:56:10,902 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:10,902 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678262170048; duration=0sec 2023-03-08 07:56:10,902 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:56:10,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/847124a51d5e4c77986fcbffad4e8da5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/b417731b8f544282ac91685d02e500a4, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c30d53b730444c1abfecdc7f8873f432, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/1af8259947ae444ba09d5160c33cbf85, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/072b99c31be5491a9f1822f42983e83f, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/dcc8cce47cd64a158e43066dd296a3e2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/cc5ab4cebd984e18a19c6464637866b2, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/e9eb9a6547a8450d9674b4894e9d115d, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/2a297909a9574b8c97d3f05a803619fa, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_] to archive 2023-03-08 07:56:10,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:10,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/847124a51d5e4c77986fcbffad4e8da5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/847124a51d5e4c77986fcbffad4e8da5 2023-03-08 07:56:10,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/b417731b8f544282ac91685d02e500a4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/b417731b8f544282ac91685d02e500a4 2023-03-08 07:56:10,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c30d53b730444c1abfecdc7f8873f432 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c30d53b730444c1abfecdc7f8873f432 2023-03-08 07:56:10,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/1af8259947ae444ba09d5160c33cbf85 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/1af8259947ae444ba09d5160c33cbf85 2023-03-08 07:56:10,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/072b99c31be5491a9f1822f42983e83f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/072b99c31be5491a9f1822f42983e83f 2023-03-08 07:56:10,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/dcc8cce47cd64a158e43066dd296a3e2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/dcc8cce47cd64a158e43066dd296a3e2 2023-03-08 07:56:10,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/cc5ab4cebd984e18a19c6464637866b2 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/cc5ab4cebd984e18a19c6464637866b2 2023-03-08 07:56:10,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/e9eb9a6547a8450d9674b4894e9d115d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/e9eb9a6547a8450d9674b4894e9d115d 2023-03-08 07:56:10,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/2a297909a9574b8c97d3f05a803619fa to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/2a297909a9574b8c97d3f05a803619fa 2023-03-08 07:56:10,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ac475693a2184bfba2adde4a3a4e19ad_SeqId_44_ 2023-03-08 07:56:10,965 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d638a25c8bfc4e59aca32202d203edb8 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d638a25c8bfc4e59aca32202d203edb8 2023-03-08 07:56:10,966 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:10,988 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dbbc7901177a4d11ade21b935b61c739 first=Optional[r00] last=Optional[r8] 2023-03-08 07:56:10,989 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dbbc7901177a4d11ade21b935b61c739}] 2023-03-08 07:56:10,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dbbc7901177a4d11ade21b935b61c739 for inclusion in 8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:56:10,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 07:56:10,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 07:56:10,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:10,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dbbc7901177a4d11ade21b935b61c739 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n4v8kl5nleo2hv8jv7omgl0b85aidcrae258696pjandj30ui8fm8kih3um540ge/family1/dbbc7901177a4d11ade21b935b61c739 2023-03-08 07:56:10,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n4v8kl5nleo2hv8jv7omgl0b85aidcrae258696pjandj30ui8fm8kih3um540ge/family1/dbbc7901177a4d11ade21b935b61c739 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_ 2023-03-08 07:56:10,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n4v8kl5nleo2hv8jv7omgl0b85aidcrae258696pjandj30ui8fm8kih3um540ge/family1/dbbc7901177a4d11ade21b935b61c739 into 8edf959f1d606a2cc1723c3487e05ae1/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_ - updating store file list. 2023-03-08 07:56:11,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_ into 8edf959f1d606a2cc1723c3487e05ae1/family1 2023-03-08 07:56:11,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n4v8kl5nleo2hv8jv7omgl0b85aidcrae258696pjandj30ui8fm8kih3um540ge/family1/dbbc7901177a4d11ade21b935b61c739 into 8edf959f1d606a2cc1723c3487e05ae1/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_) 2023-03-08 07:56:11,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n4v8kl5nleo2hv8jv7omgl0b85aidcrae258696pjandj30ui8fm8kih3um540ge/family1/dbbc7901177a4d11ade21b935b61c739 2023-03-08 07:56:11,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 07:56:11,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 07:56:11,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(1912): 8edf959f1d606a2cc1723c3487e05ae1/family1 is initiating minor compaction (all files) 2023-03-08 07:56:11,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@d7c12ab; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:56:11,002 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8edf959f1d606a2cc1723c3487e05ae1/family1 in testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:11,002 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/8778ca589d36424f8ce78b7a89f90e32, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ecaeef0337104dd1875ddb991ad64d57, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp, totalSize=344.3 K 2023-03-08 07:56:11,003 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 8778ca589d36424f8ce78b7a89f90e32, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839991 2023-03-08 07:56:11,003 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting ecaeef0337104dd1875ddb991ad64d57, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239991 2023-03-08 07:56:11,003 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting 98643b9418844e7d9c453eea9a3c5777_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965239991 2023-03-08 07:56:11,004 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] compactions.Compactor(207): Compacting c4b50be50fff47f99d205abefedf302d_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014839991 2023-03-08 07:56:11,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:11,004 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:11,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120a6bc0b077a72429e9bc001ecaf0b2160 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120a6bc0b077a72429e9bc001ecaf0b2160 2023-03-08 07:56:11,006 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128be3c49ef85f94c61902b9e580b0c4f47 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128be3c49ef85f94c61902b9e580b0c4f47 2023-03-08 07:56:11,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511303d6d263ee25346079088d300ad377eb9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511303d6d263ee25346079088d300ad377eb9 2023-03-08 07:56:11,008 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 07:56:11,008 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 07:56:11,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:11,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:11,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 07:56:11,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 07:56:11,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:56:11,011 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 07:56:11,012 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8410 sec 2023-03-08 07:56:11,029 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8edf959f1d606a2cc1723c3487e05ae1#family1#compaction#166 average throughput is 10.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:11,206 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=261 (was 248) - Thread LEAK? -, OpenFileDescriptor=652 (was 622) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=161 (was 168), ProcessCount=170 (was 170), AvailableMemoryMB=7427 (was 7441) 2023-03-08 07:56:11,218 INFO [Listener at localhost/42999] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=261, OpenFileDescriptor=652, MaxFileDescriptor=60000, SystemLoadAverage=161, ProcessCount=170, AvailableMemoryMB=7426 2023-03-08 07:56:11,237 DEBUG [Listener at localhost/42999] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-08 07:56:11,290 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x80\xCE\x89\xF9\x7FW\xEAZ\xBF\x96V\xD3\xB2\xDF\x82\x12"\x10} &\xEB%#\x9B\xCC\xCB\x80\xE8F\x11\xA0\x1F\xD6*\x10S$\xDE\x08\xA7\xEBn\xEF\xD6\xAE\xA1=\x15|\x1CK2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 07:56:11,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-08 07:56:11,292 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 07:56:11,292 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-08 07:56:11,293 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 07:56:11,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af empty. 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b empty. 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d empty. 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:11,295 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:11,296 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-08 07:56:11,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 07:56:11,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 07:56:11,702 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-08 07:56:11,704 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 19bdaae25e528e021ec383cd49c4c05d, NAME => 'testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x80\xCE\x89\xF9\x7FW\xEAZ\xBF\x96V\xD3\xB2\xDF\x82\x12"\x10} &\xEB%#\x9B\xCC\xCB\x80\xE8F\x11\xA0\x1F\xD6*\x10S$\xDE\x08\xA7\xEBn\xEF\xD6\xAE\xA1=\x15|\x1CK2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:56:11,704 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 86e34d59ea54b118fb72d7e3271d969b, NAME => 'testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x80\xCE\x89\xF9\x7FW\xEAZ\xBF\x96V\xD3\xB2\xDF\x82\x12"\x10} &\xEB%#\x9B\xCC\xCB\x80\xE8F\x11\xA0\x1F\xD6*\x10S$\xDE\x08\xA7\xEBn\xEF\xD6\xAE\xA1=\x15|\x1CK2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:56:11,704 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => d5966061f30112fde4757a0c8d47e8af, NAME => 'testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x80\xCE\x89\xF9\x7FW\xEAZ\xBF\x96V\xD3\xB2\xDF\x82\x12"\x10} &\xEB%#\x9B\xCC\xCB\x80\xE8F\x11\xA0\x1F\xD6*\x10S$\xDE\x08\xA7\xEBn\xEF\xD6\xAE\xA1=\x15|\x1CK2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/.tmp 2023-03-08 07:56:11,712 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:56:11,712 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 19bdaae25e528e021ec383cd49c4c05d, disabling compactions & flushes 2023-03-08 07:56:11,712 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:11,712 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:11,712 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. after waiting 0 ms 2023-03-08 07:56:11,712 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:11,712 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:11,712 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:11,713 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:56:11,713 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 86e34d59ea54b118fb72d7e3271d969b, disabling compactions & flushes 2023-03-08 07:56:11,713 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:11,713 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:11,714 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. after waiting 0 ms 2023-03-08 07:56:11,714 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:11,714 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:11,714 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:11,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 07:56:12,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:56:12,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing d5966061f30112fde4757a0c8d47e8af, disabling compactions & flushes 2023-03-08 07:56:12,113 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. after waiting 0 ms 2023-03-08 07:56:12,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,113 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:12,115 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 07:56:12,116 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678262172116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262172116"}]},"ts":"1678262172116"} 2023-03-08 07:56:12,116 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262172116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262172116"}]},"ts":"1678262172116"} 2023-03-08 07:56:12,116 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262172116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678262172116"}]},"ts":"1678262172116"} 2023-03-08 07:56:12,118 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 07:56:12,118 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 07:56:12,119 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262172118"}]},"ts":"1678262172118"} 2023-03-08 07:56:12,119 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-08 07:56:12,122 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d5966061f30112fde4757a0c8d47e8af, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=19bdaae25e528e021ec383cd49c4c05d, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=86e34d59ea54b118fb72d7e3271d969b, ASSIGN}] 2023-03-08 07:56:12,124 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=19bdaae25e528e021ec383cd49c4c05d, ASSIGN 2023-03-08 07:56:12,124 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d5966061f30112fde4757a0c8d47e8af, ASSIGN 2023-03-08 07:56:12,124 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=86e34d59ea54b118fb72d7e3271d969b, ASSIGN 2023-03-08 07:56:12,124 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d5966061f30112fde4757a0c8d47e8af, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:56:12,124 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=86e34d59ea54b118fb72d7e3271d969b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:56:12,124 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=19bdaae25e528e021ec383cd49c4c05d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33669,1678262016084; forceNewPlan=false, retain=false 2023-03-08 07:56:12,261 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/.tmp/family1/3c1c6af19dfc4770bc0448af1b8bdf6d as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/3c1c6af19dfc4770bc0448af1b8bdf6d 2023-03-08 07:56:12,265 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 8edf959f1d606a2cc1723c3487e05ae1/family1 of 8edf959f1d606a2cc1723c3487e05ae1 into 3c1c6af19dfc4770bc0448af1b8bdf6d(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:56:12,265 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:12,265 INFO [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1678262171002; duration=1sec 2023-03-08 07:56:12,265 DEBUG [RS:0;jenkins-hbase4:33669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 07:56:12,276 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=d5966061f30112fde4757a0c8d47e8af, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:12,276 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=86e34d59ea54b118fb72d7e3271d969b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:12,276 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=19bdaae25e528e021ec383cd49c4c05d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:12,276 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262172276"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262172276"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262172276"}]},"ts":"1678262172276"} 2023-03-08 07:56:12,276 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678262172276"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262172276"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262172276"}]},"ts":"1678262172276"} 2023-03-08 07:56:12,276 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262172276"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678262172276"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678262172276"}]},"ts":"1678262172276"} 2023-03-08 07:56:12,278 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure d5966061f30112fde4757a0c8d47e8af, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:56:12,278 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure 19bdaae25e528e021ec383cd49c4c05d, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:56:12,278 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 86e34d59ea54b118fb72d7e3271d969b, server=jenkins-hbase4.apache.org,33669,1678262016084}] 2023-03-08 07:56:12,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/8778ca589d36424f8ce78b7a89f90e32, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ecaeef0337104dd1875ddb991ad64d57, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_] to archive 2023-03-08 07:56:12,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:12,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/8778ca589d36424f8ce78b7a89f90e32 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/8778ca589d36424f8ce78b7a89f90e32 2023-03-08 07:56:12,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ecaeef0337104dd1875ddb991ad64d57 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/ecaeef0337104dd1875ddb991ad64d57 2023-03-08 07:56:12,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/98643b9418844e7d9c453eea9a3c5777_SeqId_46_ 2023-03-08 07:56:12,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_ to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/family1/c4b50be50fff47f99d205abefedf302d_SeqId_49_ 2023-03-08 07:56:12,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 07:56:12,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:12,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 19bdaae25e528e021ec383cd49c4c05d, NAME => 'testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 07:56:12,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:56:12,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,453 INFO [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,454 DEBUG [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1 2023-03-08 07:56:12,454 DEBUG [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1 2023-03-08 07:56:12,455 INFO [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19bdaae25e528e021ec383cd49c4c05d columnFamilyName family1 2023-03-08 07:56:12,456 INFO [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] regionserver.HStore(310): Store=19bdaae25e528e021ec383cd49c4c05d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:56:12,456 INFO [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,457 DEBUG [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2 2023-03-08 07:56:12,457 DEBUG [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2 2023-03-08 07:56:12,458 INFO [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19bdaae25e528e021ec383cd49c4c05d columnFamilyName family2 2023-03-08 07:56:12,459 INFO [StoreOpener-19bdaae25e528e021ec383cd49c4c05d-1] regionserver.HStore(310): Store=19bdaae25e528e021ec383cd49c4c05d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:56:12,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:56:12,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:56:12,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 19bdaae25e528e021ec383cd49c4c05d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68579803, jitterRate=0.021918699145317078}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:12,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d., pid=121, masterSystemTime=1678262172429 2023-03-08 07:56:12,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:12,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:12,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d5966061f30112fde4757a0c8d47e8af, NAME => 'testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 07:56:12,465 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=19bdaae25e528e021ec383cd49c4c05d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:12,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:56:12,465 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678262172465"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262172465"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262172465"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262172465"}]},"ts":"1678262172465"} 2023-03-08 07:56:12,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,468 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-03-08 07:56:12,468 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure 19bdaae25e528e021ec383cd49c4c05d, server=jenkins-hbase4.apache.org,33669,1678262016084 in 188 msec 2023-03-08 07:56:12,469 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=19bdaae25e528e021ec383cd49c4c05d, ASSIGN in 346 msec 2023-03-08 07:56:12,485 INFO [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,487 DEBUG [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1 2023-03-08 07:56:12,487 DEBUG [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1 2023-03-08 07:56:12,487 INFO [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5966061f30112fde4757a0c8d47e8af columnFamilyName family1 2023-03-08 07:56:12,488 INFO [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] regionserver.HStore(310): Store=d5966061f30112fde4757a0c8d47e8af/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:56:12,488 INFO [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,489 DEBUG [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2 2023-03-08 07:56:12,489 DEBUG [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2 2023-03-08 07:56:12,489 INFO [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5966061f30112fde4757a0c8d47e8af columnFamilyName family2 2023-03-08 07:56:12,489 INFO [StoreOpener-d5966061f30112fde4757a0c8d47e8af-1] regionserver.HStore(310): Store=d5966061f30112fde4757a0c8d47e8af/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:56:12,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:56:12,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:12,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:56:12,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d5966061f30112fde4757a0c8d47e8af; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60374360, jitterRate=-0.10035192966461182}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:56:12,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:12,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af., pid=120, masterSystemTime=1678262172429 2023-03-08 07:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:12,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 86e34d59ea54b118fb72d7e3271d969b, NAME => 'testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 07:56:12,496 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=d5966061f30112fde4757a0c8d47e8af, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 07:56:12,496 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262172496"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262172496"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262172496"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262172496"}]},"ts":"1678262172496"} 2023-03-08 07:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,498 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-08 07:56:12,499 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure d5966061f30112fde4757a0c8d47e8af, server=jenkins-hbase4.apache.org,33669,1678262016084 in 220 msec 2023-03-08 07:56:12,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d5966061f30112fde4757a0c8d47e8af, ASSIGN in 377 msec 2023-03-08 07:56:12,519 INFO [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,520 DEBUG [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1 2023-03-08 07:56:12,520 DEBUG [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1 2023-03-08 07:56:12,520 INFO [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86e34d59ea54b118fb72d7e3271d969b columnFamilyName family1 2023-03-08 07:56:12,521 INFO [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] regionserver.HStore(310): Store=86e34d59ea54b118fb72d7e3271d969b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:56:12,521 INFO [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,522 DEBUG [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2 2023-03-08 07:56:12,522 DEBUG [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2 2023-03-08 07:56:12,522 INFO [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86e34d59ea54b118fb72d7e3271d969b columnFamilyName family2 2023-03-08 07:56:12,523 INFO [StoreOpener-86e34d59ea54b118fb72d7e3271d969b-1] regionserver.HStore(310): Store=86e34d59ea54b118fb72d7e3271d969b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 07:56:12,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 07:56:12,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:12,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 07:56:12,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 86e34d59ea54b118fb72d7e3271d969b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65363306, jitterRate=-0.026010841131210327}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 07:56:12,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:12,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b., pid=122, masterSystemTime=1678262172429 2023-03-08 07:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:12,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:12,529 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=86e34d59ea54b118fb72d7e3271d969b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:12,529 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678262172529"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678262172529"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678262172529"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678262172529"}]},"ts":"1678262172529"} 2023-03-08 07:56:12,531 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-08 07:56:12,531 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 86e34d59ea54b118fb72d7e3271d969b, server=jenkins-hbase4.apache.org,33669,1678262016084 in 252 msec 2023-03-08 07:56:12,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-03-08 07:56:12,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=86e34d59ea54b118fb72d7e3271d969b, ASSIGN in 409 msec 2023-03-08 07:56:12,532 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 07:56:12,532 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678262172532"}]},"ts":"1678262172532"} 2023-03-08 07:56:12,533 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-08 07:56:12,536 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 07:56:12,536 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2460 sec 2023-03-08 07:56:13,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 07:56:13,398 INFO [Listener at localhost/42999] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-08 07:56:13,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:13,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:13,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:13,406 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:13,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 07:56:13,408 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 07:56:13,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:13,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 07:56:13,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 07:56:13,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:13,408 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:13,410 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:13,410 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:13,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:13,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:13,411 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:13,411 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:13,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:13,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:13,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:13,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:13,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 07:56:13,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 07:56:13,412 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 07:56:13,412 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:13,412 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 07:56:13,412 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:13,412 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:56:13,412 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:13,412 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:13,412 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. started... 2023-03-08 07:56:13,413 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. started... 2023-03-08 07:56:13,412 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:13,413 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. started... 2023-03-08 07:56:13,413 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5966061f30112fde4757a0c8d47e8af 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:56:13,413 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86e34d59ea54b118fb72d7e3271d969b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:56:13,414 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 19bdaae25e528e021ec383cd49c4c05d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 07:56:13,461 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088b505ea389b04b3e9946262470654add to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088b505ea389b04b3e9946262470654add 2023-03-08 07:56:13,470 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303080c069cd712f840878342ef19226dbd82 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080c069cd712f840878342ef19226dbd82 2023-03-08 07:56:13,470 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308aa57de8b7f264627981cd6bee9315ec3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308aa57de8b7f264627981cd6bee9315ec3 2023-03-08 07:56:13,488 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/239ce4ae31ab466f825bc93bef3233cc 2023-03-08 07:56:13,490 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/d9f7218f7f7c4ce9a8c0f94d16139991 2023-03-08 07:56:13,531 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308f5b55867882d458481b908d091a4f6c0 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308f5b55867882d458481b908d091a4f6c0 2023-03-08 07:56:13,888 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/d746db99d9674b49a32e825f066590b4 2023-03-08 07:56:13,931 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303088c49eea0222a4505a2b45e35542b9396 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303088c49eea0222a4505a2b45e35542b9396 2023-03-08 07:56:13,936 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/3a9ffeba2ceb46978b7a26dd171fe4fa 2023-03-08 07:56:13,940 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/239ce4ae31ab466f825bc93bef3233cc as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/239ce4ae31ab466f825bc93bef3233cc 2023-03-08 07:56:13,975 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/239ce4ae31ab466f825bc93bef3233cc, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 07:56:13,976 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/3a9ffeba2ceb46978b7a26dd171fe4fa as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/3a9ffeba2ceb46978b7a26dd171fe4fa 2023-03-08 07:56:13,979 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/3a9ffeba2ceb46978b7a26dd171fe4fa, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:56:13,979 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 19bdaae25e528e021ec383cd49c4c05d in 565ms, sequenceid=5, compaction requested=false 2023-03-08 07:56:13,979 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-08 07:56:13,980 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:13,980 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:13,980 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:56:14,321 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308ddd5989a73d643f3a60caeb79905dc32 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308ddd5989a73d643f3a60caeb79905dc32 2023-03-08 07:56:14,335 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/d3e29247ba104dd29d83d3671253b873 2023-03-08 07:56:14,338 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/d9f7218f7f7c4ce9a8c0f94d16139991 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/d9f7218f7f7c4ce9a8c0f94d16139991 2023-03-08 07:56:14,362 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/d9f7218f7f7c4ce9a8c0f94d16139991, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 07:56:14,363 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/d3e29247ba104dd29d83d3671253b873 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/d3e29247ba104dd29d83d3671253b873 2023-03-08 07:56:14,366 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/d3e29247ba104dd29d83d3671253b873, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:56:14,367 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d5966061f30112fde4757a0c8d47e8af in 954ms, sequenceid=5, compaction requested=false 2023-03-08 07:56:14,367 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:14,367 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:14,368 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:56:14,726 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9 2023-03-08 07:56:14,730 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/d746db99d9674b49a32e825f066590b4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/d746db99d9674b49a32e825f066590b4 2023-03-08 07:56:14,756 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/d746db99d9674b49a32e825f066590b4, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 07:56:14,757 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9 2023-03-08 07:56:14,760 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 07:56:14,760 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 86e34d59ea54b118fb72d7e3271d969b in 1347ms, sequenceid=5, compaction requested=false 2023-03-08 07:56:14,760 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:14,760 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:14,760 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:56:14,760 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:56:14,761 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,761 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 07:56:14,761 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 07:56:14,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,765 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:14,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:14,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,765 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:14,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 07:56:14,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@781e474d[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:14,767 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 07:56:14,767 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,768 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 07:56:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 07:56:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:14,768 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,768 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:14,769 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 07:56:14,769 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:14,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,769 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 07:56:14,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:14,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:14,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 07:56:14,771 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 07:56:14,771 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:14,771 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:14,771 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 07:56:14,771 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:14,773 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,773 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:14,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:14,773 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:14,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:14,774 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,775 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,776 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,776 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:14,776 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:14,776 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:14,776 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:14,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:14,776 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 07:56:14,777 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:14,776 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:14,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:14,777 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,777 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,777 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:14,777 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,777 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:14,777 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:14,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:14,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:14,877 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:14,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:14,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:14,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:14,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:14,881 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:14,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:14,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 07:56:14,883 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 07:56:14,883 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:14,883 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 07:56:14,883 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 07:56:14,884 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,884 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:14,885 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:14,885 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:14,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:14,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:14,886 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:14,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:14,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:14,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:14,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:14,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 07:56:14,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 07:56:14,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 07:56:14,887 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 07:56:14,887 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:14,887 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 07:56:14,887 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:14,887 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. started... 2023-03-08 07:56:14,887 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:56:14,887 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:14,887 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:14,887 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:14,887 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5966061f30112fde4757a0c8d47e8af 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:56:14,887 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. started... 2023-03-08 07:56:14,887 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. started... 2023-03-08 07:56:14,887 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 19bdaae25e528e021ec383cd49c4c05d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:56:14,887 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86e34d59ea54b118fb72d7e3271d969b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 07:56:14,930 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030864d6f46f88334c66b3d3019e29ecbb1e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d6f46f88334c66b3d3019e29ecbb1e 2023-03-08 07:56:14,950 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085e761be0854345f887ce2f6e6343c52c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085e761be0854345f887ce2f6e6343c52c 2023-03-08 07:56:15,351 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/3c0da674bfc2411687651d9cd9866711 2023-03-08 07:56:15,351 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308724da24191a44c15b02c2f81ed6f4478 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308724da24191a44c15b02c2f81ed6f4478 2023-03-08 07:56:15,372 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/9bb360071a134eb6827508736f7fed68 2023-03-08 07:56:15,375 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/8e91d27ae11c40ca9cb180ae2541742c 2023-03-08 07:56:15,400 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303084940cf1306ad4b24bfd223323949998d to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303084940cf1306ad4b24bfd223323949998d 2023-03-08 07:56:15,814 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/b6cb79a53830473aba5e4c1e1ebec6cb 2023-03-08 07:56:15,818 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/3c0da674bfc2411687651d9cd9866711 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/3c0da674bfc2411687651d9cd9866711 2023-03-08 07:56:15,828 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308014d1f3a636146988709759cac71dadf to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308014d1f3a636146988709759cac71dadf 2023-03-08 07:56:15,835 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b14d72024c4a4c80b19f18a9a70b5dae to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308b14d72024c4a4c80b19f18a9a70b5dae 2023-03-08 07:56:15,843 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/3c0da674bfc2411687651d9cd9866711, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 07:56:15,843 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/b6cb79a53830473aba5e4c1e1ebec6cb as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/b6cb79a53830473aba5e4c1e1ebec6cb 2023-03-08 07:56:15,846 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/b6cb79a53830473aba5e4c1e1ebec6cb, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:56:15,847 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d5966061f30112fde4757a0c8d47e8af in 960ms, sequenceid=9, compaction requested=false 2023-03-08 07:56:15,847 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:15,847 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:15,847 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:56:16,233 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/734ed4a6524f4493a599586e1201ead1 2023-03-08 07:56:16,236 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/9bb360071a134eb6827508736f7fed68 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/9bb360071a134eb6827508736f7fed68 2023-03-08 07:56:16,239 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/c75e45907d104dbd88333ca03d618312 2023-03-08 07:56:16,242 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/8e91d27ae11c40ca9cb180ae2541742c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/8e91d27ae11c40ca9cb180ae2541742c 2023-03-08 07:56:16,261 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/9bb360071a134eb6827508736f7fed68, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 07:56:16,261 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/734ed4a6524f4493a599586e1201ead1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/734ed4a6524f4493a599586e1201ead1 2023-03-08 07:56:16,264 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/734ed4a6524f4493a599586e1201ead1, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:56:16,265 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 19bdaae25e528e021ec383cd49c4c05d in 1378ms, sequenceid=9, compaction requested=false 2023-03-08 07:56:16,265 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:16,265 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:16,265 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:56:16,265 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/8e91d27ae11c40ca9cb180ae2541742c, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 07:56:16,266 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/c75e45907d104dbd88333ca03d618312 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/c75e45907d104dbd88333ca03d618312 2023-03-08 07:56:16,269 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/c75e45907d104dbd88333ca03d618312, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 07:56:16,269 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 86e34d59ea54b118fb72d7e3271d969b in 1382ms, sequenceid=9, compaction requested=false 2023-03-08 07:56:16,269 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:16,269 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:16,269 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:56:16,269 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:56:16,270 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,270 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 07:56:16,270 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 07:56:16,271 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,271 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:16,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:16,271 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,271 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:16,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:16,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:16,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:16,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:16,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 07:56:16,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4347902c[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:16,272 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 07:56:16,272 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,275 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,275 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 07:56:16,275 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 07:56:16,275 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:16,275 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,275 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:16,277 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 07:56:16,277 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:16,277 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,277 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 07:56:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:16,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 07:56:16,279 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 07:56:16,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:16,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:16,279 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 07:56:16,279 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:16,281 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,281 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,281 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:16,281 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:16,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:16,282 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:16,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:16,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:16,284 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 07:56:16,284 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:16,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:16,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:16,284 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,285 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:16,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 07:56:16,285 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,284 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:16,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:16,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:16,285 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:16,385 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:16,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:16,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:16,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:16,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:16,389 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:16,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:16,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 07:56:16,391 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 07:56:16,391 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:16,391 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 07:56:16,391 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 07:56:16,391 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,391 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:16,393 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,393 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:16,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:16,393 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:16,393 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:16,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:16,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:16,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:16,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:16,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 07:56:16,394 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 07:56:16,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 07:56:16,394 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 07:56:16,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:16,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 07:56:16,395 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:16,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:56:16,395 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:16,395 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:16,395 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. started... 2023-03-08 07:56:16,395 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:16,395 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. started... 2023-03-08 07:56:16,395 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. started... 2023-03-08 07:56:16,395 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 19bdaae25e528e021ec383cd49c4c05d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:56:16,395 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5966061f30112fde4757a0c8d47e8af 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:56:16,395 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86e34d59ea54b118fb72d7e3271d969b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 07:56:16,420 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:56:16,420 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b4be601 to 127.0.0.1:52387 2023-03-08 07:56:16,420 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:16,421 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:56:16,421 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x23871a53 to 127.0.0.1:52387 2023-03-08 07:56:16,421 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:16,457 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030899150c80546a4441a13758a5d458914e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030899150c80546a4441a13758a5d458914e 2023-03-08 07:56:16,857 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308fdf0f4d66c8e4339a0e42d41916f4601 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308fdf0f4d66c8e4339a0e42d41916f4601 2023-03-08 07:56:16,868 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a451a2bb820946dabe588369e0b71bd6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a451a2bb820946dabe588369e0b71bd6 2023-03-08 07:56:16,873 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/57bc63f1634741229f1d1ccf7132e132 2023-03-08 07:56:16,890 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/5d7e3a59d1c1494b8532af5183ed2627 2023-03-08 07:56:16,907 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e07dc749f65f47ea8ceb7b0b7037c60a to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308e07dc749f65f47ea8ceb7b0b7037c60a 2023-03-08 07:56:16,936 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308dcc02743acae42549d9b6a8b5544012c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308dcc02743acae42549d9b6a8b5544012c 2023-03-08 07:56:16,945 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/56efe9c4812c49f1943161277e09e7a1 2023-03-08 07:56:16,948 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/5d7e3a59d1c1494b8532af5183ed2627 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/5d7e3a59d1c1494b8532af5183ed2627 2023-03-08 07:56:16,973 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/5d7e3a59d1c1494b8532af5183ed2627, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 07:56:16,973 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/56efe9c4812c49f1943161277e09e7a1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/56efe9c4812c49f1943161277e09e7a1 2023-03-08 07:56:16,977 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/56efe9c4812c49f1943161277e09e7a1, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:56:16,978 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d5966061f30112fde4757a0c8d47e8af in 583ms, sequenceid=13, compaction requested=true 2023-03-08 07:56:16,978 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:16,978 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:16,978 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:56:17,275 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/8770ae2705de4cf7a06e987c7061f1ef 2023-03-08 07:56:17,320 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/37d2a1d0512843febf1ec70313d16707 2023-03-08 07:56:17,324 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/57bc63f1634741229f1d1ccf7132e132 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/57bc63f1634741229f1d1ccf7132e132 2023-03-08 07:56:17,328 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303084c51bf3817704f53a87d020dba68790e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303084c51bf3817704f53a87d020dba68790e 2023-03-08 07:56:17,337 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/8caa8d930bed4340bdb43999f7c531df 2023-03-08 07:56:17,341 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/8770ae2705de4cf7a06e987c7061f1ef as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8770ae2705de4cf7a06e987c7061f1ef 2023-03-08 07:56:17,351 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/57bc63f1634741229f1d1ccf7132e132, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 07:56:17,352 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/37d2a1d0512843febf1ec70313d16707 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/37d2a1d0512843febf1ec70313d16707 2023-03-08 07:56:17,356 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/37d2a1d0512843febf1ec70313d16707, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:56:17,357 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 86e34d59ea54b118fb72d7e3271d969b in 962ms, sequenceid=13, compaction requested=true 2023-03-08 07:56:17,357 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:17,357 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:17,357 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:56:17,365 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8770ae2705de4cf7a06e987c7061f1ef, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 07:56:17,366 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/8caa8d930bed4340bdb43999f7c531df as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/8caa8d930bed4340bdb43999f7c531df 2023-03-08 07:56:17,371 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/8caa8d930bed4340bdb43999f7c531df, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 07:56:17,371 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 19bdaae25e528e021ec383cd49c4c05d in 976ms, sequenceid=13, compaction requested=true 2023-03-08 07:56:17,371 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:17,371 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:17,371 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:56:17,371 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:56:17,372 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,372 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 07:56:17,372 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 07:56:17,373 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,373 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:17,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:17,373 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,373 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:17,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:17,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:17,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:17,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:17,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 07:56:17,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44e1c9f9[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:17,375 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 07:56:17,375 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,376 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,376 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 07:56:17,376 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 07:56:17,376 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:17,376 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,376 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:17,377 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,377 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 07:56:17,377 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:17,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,378 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 07:56:17,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:17,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:17,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:17,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:17,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:17,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:17,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:17,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:17,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 07:56:17,380 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 07:56:17,380 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:17,380 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:17,380 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 07:56:17,380 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:17,383 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,383 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:17,384 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:17,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:17,384 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:17,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:17,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:17,385 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:17,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:17,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:17,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:17,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:17,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 07:56:17,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:17,387 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:17,387 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:17,388 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:17,388 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:17,388 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,388 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:17,488 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:17,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:17,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:17,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:17,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 07:56:17,492 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:17,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:17,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 07:56:17,494 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 07:56:17,494 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:17,494 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 07:56:17,494 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 07:56:17,494 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,494 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:17,498 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,499 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:17,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:17,499 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:17,499 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:17,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:17,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 07:56:17,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 07:56:17,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 07:56:17,500 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 07:56:17,500 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:17,500 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 07:56:17,500 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:17,500 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. started... 2023-03-08 07:56:17,500 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:56:17,500 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:17,500 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5966061f30112fde4757a0c8d47e8af 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:56:17,500 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:17,500 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:17,500 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. started... 2023-03-08 07:56:17,500 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. started... 2023-03-08 07:56:17,501 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 19bdaae25e528e021ec383cd49c4c05d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:56:17,501 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86e34d59ea54b118fb72d7e3271d969b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 07:56:17,548 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303081f5c1e263c124b71baae60a9447280b9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303081f5c1e263c124b71baae60a9447280b9 2023-03-08 07:56:17,556 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e0e105fe53f74905b66326acbc5ba99b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308e0e105fe53f74905b66326acbc5ba99b 2023-03-08 07:56:17,568 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/64f12bf27fe14747a5c7899553809725 2023-03-08 07:56:17,603 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030821130955955846cca648b951afd89359 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023030821130955955846cca648b951afd89359 2023-03-08 07:56:17,615 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/e26925ce394340ff9d3526de68f84025 2023-03-08 07:56:17,618 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/64f12bf27fe14747a5c7899553809725 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/64f12bf27fe14747a5c7899553809725 2023-03-08 07:56:17,641 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/64f12bf27fe14747a5c7899553809725, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 07:56:17,642 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/e26925ce394340ff9d3526de68f84025 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/e26925ce394340ff9d3526de68f84025 2023-03-08 07:56:17,646 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/e26925ce394340ff9d3526de68f84025, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:56:17,646 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 19bdaae25e528e021ec383cd49c4c05d in 145ms, sequenceid=17, compaction requested=true 2023-03-08 07:56:17,646 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:17,646 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:17,646 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:56:17,941 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303083eb4674dd8ae44be977226ddd961d854 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303083eb4674dd8ae44be977226ddd961d854 2023-03-08 07:56:17,955 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5 2023-03-08 07:56:17,971 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 07:56:17,977 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/157051b517874a869087c0a8323b24e4 2023-03-08 07:56:18,031 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030866827ef8b3bd402f977dd4052c417fa7 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023030866827ef8b3bd402f977dd4052c417fa7 2023-03-08 07:56:18,036 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ce39b6b6cc664a30ab1a28060ee4f340 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308ce39b6b6cc664a30ab1a28060ee4f340 2023-03-08 07:56:18,442 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/7287113830254c63842a8b7dc44be953 2023-03-08 07:56:18,445 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5 2023-03-08 07:56:18,453 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/c1e2bbe2fdeb4177b2c1910f281a64d1 2023-03-08 07:56:18,456 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/157051b517874a869087c0a8323b24e4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/157051b517874a869087c0a8323b24e4 2023-03-08 07:56:18,468 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 07:56:18,469 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/7287113830254c63842a8b7dc44be953 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/7287113830254c63842a8b7dc44be953 2023-03-08 07:56:18,473 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/7287113830254c63842a8b7dc44be953, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:56:18,473 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 86e34d59ea54b118fb72d7e3271d969b in 972ms, sequenceid=17, compaction requested=true 2023-03-08 07:56:18,473 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:18,473 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:18,473 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:56:18,478 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/157051b517874a869087c0a8323b24e4, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 07:56:18,478 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/c1e2bbe2fdeb4177b2c1910f281a64d1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/c1e2bbe2fdeb4177b2c1910f281a64d1 2023-03-08 07:56:18,481 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/c1e2bbe2fdeb4177b2c1910f281a64d1, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 07:56:18,482 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d5966061f30112fde4757a0c8d47e8af in 982ms, sequenceid=17, compaction requested=true 2023-03-08 07:56:18,482 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:18,482 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:18,482 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:56:18,482 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:56:18,482 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,482 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 07:56:18,482 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 07:56:18,484 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,484 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:18,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:18,484 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,484 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:18,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:18,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:18,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:18,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:18,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 07:56:18,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b1d3d4f[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:18,486 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 07:56:18,486 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,487 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,487 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 07:56:18,488 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 07:56:18,488 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:18,488 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,488 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:18,489 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 07:56:18,489 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:18,489 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,489 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 07:56:18,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:18,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:18,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:18,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:18,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:18,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:18,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:18,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:18,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 07:56:18,491 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 07:56:18,491 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:18,491 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:18,491 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 07:56:18,491 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:18,494 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,494 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:18,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,494 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:18,495 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:18,496 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:18,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:18,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:18,498 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:18,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:18,498 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 07:56:18,498 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:18,498 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:18,499 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:18,499 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:18,599 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:18,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:18,610 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:56:18,618 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:56:18,631 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:56:18,632 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:56:18,740 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 07:56:18,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 07:56:18,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 07:56:18,742 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 07:56:18,742 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:18,742 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 07:56:18,742 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 07:56:18,743 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,743 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:18,744 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:18,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:18,744 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:18,744 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:18,745 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 07:56:18,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:18,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:18,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 07:56:18,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:18,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 07:56:18,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 07:56:18,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 07:56:18,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 07:56:18,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 07:56:18,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 07:56:18,746 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:18,746 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. started... 2023-03-08 07:56:18,746 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5966061f30112fde4757a0c8d47e8af 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:56:18,748 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:18,748 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. started... 2023-03-08 07:56:18,748 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86e34d59ea54b118fb72d7e3271d969b 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:56:18,756 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 07:56:18,756 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 07:56:18,756 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:18,756 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. started... 2023-03-08 07:56:18,756 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 19bdaae25e528e021ec383cd49c4c05d 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 07:56:18,817 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/2fa884e6ce7341a8b011aa69854c8f14 2023-03-08 07:56:18,840 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fa884e6ce7341a8b011aa69854c8f14 2023-03-08 07:56:18,851 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/57bf2b67d4f945eb979ae59ffc41e9da 2023-03-08 07:56:18,854 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57bf2b67d4f945eb979ae59ffc41e9da 2023-03-08 07:56:18,855 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/2fa884e6ce7341a8b011aa69854c8f14 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/2fa884e6ce7341a8b011aa69854c8f14 2023-03-08 07:56:18,881 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fa884e6ce7341a8b011aa69854c8f14 2023-03-08 07:56:18,881 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/2fa884e6ce7341a8b011aa69854c8f14, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 07:56:18,882 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/57bf2b67d4f945eb979ae59ffc41e9da as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/57bf2b67d4f945eb979ae59ffc41e9da 2023-03-08 07:56:18,886 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57bf2b67d4f945eb979ae59ffc41e9da 2023-03-08 07:56:18,886 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/57bf2b67d4f945eb979ae59ffc41e9da, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 07:56:18,886 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 19bdaae25e528e021ec383cd49c4c05d in 130ms, sequenceid=23, compaction requested=true 2023-03-08 07:56:18,886 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:18,886 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:18,887 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 07:56:19,217 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/6d285c0a478a422895d7b3a497a3742f 2023-03-08 07:56:19,226 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/ccf522cdf071486791fcbc1dfa7c452f 2023-03-08 07:56:19,247 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d285c0a478a422895d7b3a497a3742f 2023-03-08 07:56:19,263 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccf522cdf071486791fcbc1dfa7c452f 2023-03-08 07:56:19,688 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/2656d73228074e0fb60bac52b8496e2b 2023-03-08 07:56:19,692 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2656d73228074e0fb60bac52b8496e2b 2023-03-08 07:56:19,693 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/6d285c0a478a422895d7b3a497a3742f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/6d285c0a478a422895d7b3a497a3742f 2023-03-08 07:56:19,696 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/be599048ff5d43c8b469b617f6c15d44 2023-03-08 07:56:19,699 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be599048ff5d43c8b469b617f6c15d44 2023-03-08 07:56:19,699 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/ccf522cdf071486791fcbc1dfa7c452f as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/ccf522cdf071486791fcbc1dfa7c452f 2023-03-08 07:56:19,724 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d285c0a478a422895d7b3a497a3742f 2023-03-08 07:56:19,724 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/6d285c0a478a422895d7b3a497a3742f, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 07:56:19,732 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/2656d73228074e0fb60bac52b8496e2b as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/2656d73228074e0fb60bac52b8496e2b 2023-03-08 07:56:19,733 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccf522cdf071486791fcbc1dfa7c452f 2023-03-08 07:56:19,733 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/ccf522cdf071486791fcbc1dfa7c452f, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 07:56:19,734 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/be599048ff5d43c8b469b617f6c15d44 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/be599048ff5d43c8b469b617f6c15d44 2023-03-08 07:56:19,744 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2656d73228074e0fb60bac52b8496e2b 2023-03-08 07:56:19,744 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be599048ff5d43c8b469b617f6c15d44 2023-03-08 07:56:19,744 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/2656d73228074e0fb60bac52b8496e2b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 07:56:19,744 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/be599048ff5d43c8b469b617f6c15d44, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 07:56:19,745 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d5966061f30112fde4757a0c8d47e8af in 999ms, sequenceid=23, compaction requested=true 2023-03-08 07:56:19,745 INFO [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 86e34d59ea54b118fb72d7e3271d969b in 997ms, sequenceid=23, compaction requested=true 2023-03-08 07:56:19,745 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:19,745 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:19,745 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:19,745 DEBUG [rs(jenkins-hbase4.apache.org,33669,1678262016084)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:19,745 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 07:56:19,745 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 07:56:19,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 07:56:19,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 07:56:19,746 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 07:56:19,747 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,747 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:19,747 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,747 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 07:56:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:19,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33669,1678262016084' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 07:56:19,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61332b4e[Count = 0] remaining members to acquire global barrier 2023-03-08 07:56:19,749 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 07:56:19,749 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,750 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 07:56:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 07:56:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33669,1678262016084' in zk 2023-03-08 07:56:19,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,751 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 07:56:19,758 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 07:56:19,758 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,758 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:19,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:19,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:19,758 DEBUG [member: 'jenkins-hbase4.apache.org,33669,1678262016084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 07:56:19,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:19,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:19,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:19,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:19,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:19,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33669,1678262016084': 2023-03-08 07:56:19,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33669,1678262016084' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 07:56:19,761 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 07:56:19,761 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 07:56:19,761 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 07:56:19,761 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 07:56:19,762 INFO [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 07:56:19,763 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,763 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:19,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,763 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 07:56:19,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 07:56:19,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:19,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 07:56:19,765 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 07:56:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:19,766 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 07:56:19,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 07:56:19,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 07:56:19,769 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 07:56:19,769 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 07:56:19,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 07:56:19,769 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 07:56:19,769 DEBUG [(jenkins-hbase4.apache.org,35671,1678262014900)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 07:56:19,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 07:56:19,769 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 07:56:19,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 07:56:19,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 07:56:19,870 DEBUG [Listener at localhost/42999] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 07:56:19,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35671] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 07:56:19,871 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:56:19,871 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): d5966061f30112fde4757a0c8d47e8af/family1 is initiating major compaction (all files) 2023-03-08 07:56:19,871 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:56:19,872 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:56:19,872 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of d5966061f30112fde4757a0c8d47e8af/family1 in testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:19,872 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/d9f7218f7f7c4ce9a8c0f94d16139991, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/3c0da674bfc2411687651d9cd9866711, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/5d7e3a59d1c1494b8532af5183ed2627, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/157051b517874a869087c0a8323b24e4, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/6d285c0a478a422895d7b3a497a3742f] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp, totalSize=27.1 K 2023-03-08 07:56:19,873 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting d9f7218f7f7c4ce9a8c0f94d16139991, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262173403 2023-03-08 07:56:19,873 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 3c0da674bfc2411687651d9cd9866711, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262174879 2023-03-08 07:56:19,873 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 5d7e3a59d1c1494b8532af5183ed2627, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262176387 2023-03-08 07:56:19,874 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 157051b517874a869087c0a8323b24e4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262177490 2023-03-08 07:56:19,874 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 6d285c0a478a422895d7b3a497a3742f, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:56:19,896 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): d5966061f30112fde4757a0c8d47e8af#family1#compaction#197 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:19,950 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del 2023-03-08 07:56:19,950 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del 2023-03-08 07:56:20,414 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da529f57f3f146959b19c6a156014907 2023-03-08 07:56:20,415 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family1/da529f57f3f146959b19c6a156014907 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/da529f57f3f146959b19c6a156014907 2023-03-08 07:56:20,437 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da529f57f3f146959b19c6a156014907 2023-03-08 07:56:20,438 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d5966061f30112fde4757a0c8d47e8af/family1 of d5966061f30112fde4757a0c8d47e8af into da529f57f3f146959b19c6a156014907(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:56:20,438 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:20,438 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:56:20,439 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): d5966061f30112fde4757a0c8d47e8af/family2 is initiating major compaction (all files) 2023-03-08 07:56:20,439 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:56:20,439 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:56:20,441 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of d5966061f30112fde4757a0c8d47e8af/family2 in testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:20,441 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/d3e29247ba104dd29d83d3671253b873, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/b6cb79a53830473aba5e4c1e1ebec6cb, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/56efe9c4812c49f1943161277e09e7a1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/c1e2bbe2fdeb4177b2c1910f281a64d1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/2656d73228074e0fb60bac52b8496e2b] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp, totalSize=25.4 K 2023-03-08 07:56:20,441 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting d3e29247ba104dd29d83d3671253b873, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262173403 2023-03-08 07:56:20,442 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b6cb79a53830473aba5e4c1e1ebec6cb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262174879 2023-03-08 07:56:20,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/d9f7218f7f7c4ce9a8c0f94d16139991, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/3c0da674bfc2411687651d9cd9866711, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/5d7e3a59d1c1494b8532af5183ed2627, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/157051b517874a869087c0a8323b24e4, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/6d285c0a478a422895d7b3a497a3742f] to archive 2023-03-08 07:56:20,442 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 56efe9c4812c49f1943161277e09e7a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262176387 2023-03-08 07:56:20,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:20,443 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting c1e2bbe2fdeb4177b2c1910f281a64d1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262177490 2023-03-08 07:56:20,443 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 2656d73228074e0fb60bac52b8496e2b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:56:20,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/d9f7218f7f7c4ce9a8c0f94d16139991 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/d9f7218f7f7c4ce9a8c0f94d16139991 2023-03-08 07:56:20,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/3c0da674bfc2411687651d9cd9866711 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/3c0da674bfc2411687651d9cd9866711 2023-03-08 07:56:20,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/5d7e3a59d1c1494b8532af5183ed2627 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/5d7e3a59d1c1494b8532af5183ed2627 2023-03-08 07:56:20,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/157051b517874a869087c0a8323b24e4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/157051b517874a869087c0a8323b24e4 2023-03-08 07:56:20,448 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/6d285c0a478a422895d7b3a497a3742f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/6d285c0a478a422895d7b3a497a3742f 2023-03-08 07:56:20,451 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): d5966061f30112fde4757a0c8d47e8af#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:20,862 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303084e7183c1c5cc4ac19d841b6293ee6b06_del 2023-03-08 07:56:20,862 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303084e7183c1c5cc4ac19d841b6293ee6b06_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303084e7183c1c5cc4ac19d841b6293ee6b06_del 2023-03-08 07:56:21,675 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df2431ab271e4806b3a4b38e645ff98c 2023-03-08 07:56:21,676 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/.tmp/family2/df2431ab271e4806b3a4b38e645ff98c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/df2431ab271e4806b3a4b38e645ff98c 2023-03-08 07:56:21,680 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df2431ab271e4806b3a4b38e645ff98c 2023-03-08 07:56:21,680 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d5966061f30112fde4757a0c8d47e8af/family2 of d5966061f30112fde4757a0c8d47e8af into df2431ab271e4806b3a4b38e645ff98c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:56:21,681 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:21,681 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:56:21,681 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 86e34d59ea54b118fb72d7e3271d969b/family1 is initiating major compaction (all files) 2023-03-08 07:56:21,681 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:56:21,681 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:56:21,681 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 86e34d59ea54b118fb72d7e3271d969b/family1 in testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:21,681 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/d746db99d9674b49a32e825f066590b4, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/8e91d27ae11c40ca9cb180ae2541742c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/57bc63f1634741229f1d1ccf7132e132, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/ccf522cdf071486791fcbc1dfa7c452f] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp, totalSize=27.1 K 2023-03-08 07:56:21,682 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting d746db99d9674b49a32e825f066590b4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262173404 2023-03-08 07:56:21,682 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 8e91d27ae11c40ca9cb180ae2541742c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262174879 2023-03-08 07:56:21,682 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 57bc63f1634741229f1d1ccf7132e132, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262176387 2023-03-08 07:56:21,683 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting f7c1f4ebe0774e13bcd38bf6b3e1b8b5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262177490 2023-03-08 07:56:21,683 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting ccf522cdf071486791fcbc1dfa7c452f, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:56:21,690 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 86e34d59ea54b118fb72d7e3271d969b#family1#compaction#199 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:21,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/d3e29247ba104dd29d83d3671253b873, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/b6cb79a53830473aba5e4c1e1ebec6cb, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/56efe9c4812c49f1943161277e09e7a1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/c1e2bbe2fdeb4177b2c1910f281a64d1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/2656d73228074e0fb60bac52b8496e2b] to archive 2023-03-08 07:56:21,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:21,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/d3e29247ba104dd29d83d3671253b873 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/d3e29247ba104dd29d83d3671253b873 2023-03-08 07:56:21,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/b6cb79a53830473aba5e4c1e1ebec6cb to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/b6cb79a53830473aba5e4c1e1ebec6cb 2023-03-08 07:56:21,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/56efe9c4812c49f1943161277e09e7a1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/56efe9c4812c49f1943161277e09e7a1 2023-03-08 07:56:21,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/c1e2bbe2fdeb4177b2c1910f281a64d1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/c1e2bbe2fdeb4177b2c1910f281a64d1 2023-03-08 07:56:21,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/2656d73228074e0fb60bac52b8496e2b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family2/2656d73228074e0fb60bac52b8496e2b 2023-03-08 07:56:22,127 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del 2023-03-08 07:56:22,127 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del 2023-03-08 07:56:22,979 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 82a2715752384f6e9d0bf427021401cb 2023-03-08 07:56:22,980 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family1/82a2715752384f6e9d0bf427021401cb as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/82a2715752384f6e9d0bf427021401cb 2023-03-08 07:56:23,003 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 82a2715752384f6e9d0bf427021401cb 2023-03-08 07:56:23,004 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 86e34d59ea54b118fb72d7e3271d969b/family1 of 86e34d59ea54b118fb72d7e3271d969b into 82a2715752384f6e9d0bf427021401cb(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:56:23,004 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:23,004 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:56:23,004 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 86e34d59ea54b118fb72d7e3271d969b/family2 is initiating major compaction (all files) 2023-03-08 07:56:23,004 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:56:23,004 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:56:23,004 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 86e34d59ea54b118fb72d7e3271d969b/family2 in testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:23,005 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/c75e45907d104dbd88333ca03d618312, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/37d2a1d0512843febf1ec70313d16707, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/7287113830254c63842a8b7dc44be953, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/be599048ff5d43c8b469b617f6c15d44] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp, totalSize=25.4 K 2023-03-08 07:56:23,005 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting b67ac3d2fd81433bb0d1e1c2e4899dd9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262173404 2023-03-08 07:56:23,005 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting c75e45907d104dbd88333ca03d618312, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262174879 2023-03-08 07:56:23,006 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 37d2a1d0512843febf1ec70313d16707, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262176387 2023-03-08 07:56:23,006 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 7287113830254c63842a8b7dc44be953, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262177490 2023-03-08 07:56:23,006 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting be599048ff5d43c8b469b617f6c15d44, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:56:23,014 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 86e34d59ea54b118fb72d7e3271d969b#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:23,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/d746db99d9674b49a32e825f066590b4, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/8e91d27ae11c40ca9cb180ae2541742c, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/57bc63f1634741229f1d1ccf7132e132, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/ccf522cdf071486791fcbc1dfa7c452f] to archive 2023-03-08 07:56:23,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:23,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/d746db99d9674b49a32e825f066590b4 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/d746db99d9674b49a32e825f066590b4 2023-03-08 07:56:23,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/8e91d27ae11c40ca9cb180ae2541742c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/8e91d27ae11c40ca9cb180ae2541742c 2023-03-08 07:56:23,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/57bc63f1634741229f1d1ccf7132e132 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/57bc63f1634741229f1d1ccf7132e132 2023-03-08 07:56:23,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/f7c1f4ebe0774e13bcd38bf6b3e1b8b5 2023-03-08 07:56:23,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/ccf522cdf071486791fcbc1dfa7c452f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/ccf522cdf071486791fcbc1dfa7c452f 2023-03-08 07:56:23,427 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308eeb020d3c5724ec6bd1652b429bcefc7_del 2023-03-08 07:56:23,427 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308eeb020d3c5724ec6bd1652b429bcefc7_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308eeb020d3c5724ec6bd1652b429bcefc7_del 2023-03-08 07:56:23,841 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dddf733f2aec4d5683a0dd68f646c2f4 2023-03-08 07:56:23,842 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/.tmp/family2/dddf733f2aec4d5683a0dd68f646c2f4 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/dddf733f2aec4d5683a0dd68f646c2f4 2023-03-08 07:56:23,845 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dddf733f2aec4d5683a0dd68f646c2f4 2023-03-08 07:56:23,846 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 86e34d59ea54b118fb72d7e3271d969b/family2 of 86e34d59ea54b118fb72d7e3271d969b into dddf733f2aec4d5683a0dd68f646c2f4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:56:23,846 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:23,847 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:56:23,847 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 19bdaae25e528e021ec383cd49c4c05d/family1 is initiating major compaction (all files) 2023-03-08 07:56:23,847 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:56:23,847 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:56:23,847 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 19bdaae25e528e021ec383cd49c4c05d/family1 in testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:23,847 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/239ce4ae31ab466f825bc93bef3233cc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/9bb360071a134eb6827508736f7fed68, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8770ae2705de4cf7a06e987c7061f1ef, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/64f12bf27fe14747a5c7899553809725, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/2fa884e6ce7341a8b011aa69854c8f14] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp, totalSize=27.1 K 2023-03-08 07:56:23,848 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 239ce4ae31ab466f825bc93bef3233cc, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262173404 2023-03-08 07:56:23,848 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 9bb360071a134eb6827508736f7fed68, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262174879 2023-03-08 07:56:23,848 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 8770ae2705de4cf7a06e987c7061f1ef, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262176387 2023-03-08 07:56:23,849 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 64f12bf27fe14747a5c7899553809725, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262177490 2023-03-08 07:56:23,849 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 2fa884e6ce7341a8b011aa69854c8f14, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:56:23,856 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 19bdaae25e528e021ec383cd49c4c05d#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:23,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/c75e45907d104dbd88333ca03d618312, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/37d2a1d0512843febf1ec70313d16707, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/7287113830254c63842a8b7dc44be953, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/be599048ff5d43c8b469b617f6c15d44] to archive 2023-03-08 07:56:23,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:23,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/b67ac3d2fd81433bb0d1e1c2e4899dd9 2023-03-08 07:56:23,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/c75e45907d104dbd88333ca03d618312 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/c75e45907d104dbd88333ca03d618312 2023-03-08 07:56:23,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/37d2a1d0512843febf1ec70313d16707 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/37d2a1d0512843febf1ec70313d16707 2023-03-08 07:56:23,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/7287113830254c63842a8b7dc44be953 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/7287113830254c63842a8b7dc44be953 2023-03-08 07:56:23,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/be599048ff5d43c8b469b617f6c15d44 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family2/be599048ff5d43c8b469b617f6c15d44 2023-03-08 07:56:24,300 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del 2023-03-08 07:56:24,300 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del 2023-03-08 07:56:25,152 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8201dda59fef43808381c4884f981c58 2023-03-08 07:56:25,152 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family1/8201dda59fef43808381c4884f981c58 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8201dda59fef43808381c4884f981c58 2023-03-08 07:56:25,173 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8201dda59fef43808381c4884f981c58 2023-03-08 07:56:25,174 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 19bdaae25e528e021ec383cd49c4c05d/family1 of 19bdaae25e528e021ec383cd49c4c05d into 8201dda59fef43808381c4884f981c58(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 07:56:25,174 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:25,175 DEBUG [Listener at localhost/42999] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 07:56:25,175 DEBUG [Listener at localhost/42999] regionserver.HStore(1912): 19bdaae25e528e021ec383cd49c4c05d/family2 is initiating major compaction (all files) 2023-03-08 07:56:25,175 INFO [Listener at localhost/42999] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 07:56:25,175 INFO [Listener at localhost/42999] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 07:56:25,175 INFO [Listener at localhost/42999] regionserver.HRegion(2259): Starting compaction of 19bdaae25e528e021ec383cd49c4c05d/family2 in testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:25,175 INFO [Listener at localhost/42999] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/3a9ffeba2ceb46978b7a26dd171fe4fa, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/734ed4a6524f4493a599586e1201ead1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/8caa8d930bed4340bdb43999f7c531df, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/e26925ce394340ff9d3526de68f84025, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/57bf2b67d4f945eb979ae59ffc41e9da] into tmpdir=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp, totalSize=25.4 K 2023-03-08 07:56:25,175 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 3a9ffeba2ceb46978b7a26dd171fe4fa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678262173404 2023-03-08 07:56:25,176 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 734ed4a6524f4493a599586e1201ead1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678262174879 2023-03-08 07:56:25,176 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 8caa8d930bed4340bdb43999f7c531df, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678262176387 2023-03-08 07:56:25,176 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting e26925ce394340ff9d3526de68f84025, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678262177490 2023-03-08 07:56:25,177 DEBUG [Listener at localhost/42999] compactions.Compactor(207): Compacting 57bf2b67d4f945eb979ae59ffc41e9da, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 07:56:25,184 INFO [Listener at localhost/42999] throttle.PressureAwareThroughputController(145): 19bdaae25e528e021ec383cd49c4c05d#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 07:56:25,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/239ce4ae31ab466f825bc93bef3233cc, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/9bb360071a134eb6827508736f7fed68, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8770ae2705de4cf7a06e987c7061f1ef, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/64f12bf27fe14747a5c7899553809725, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/2fa884e6ce7341a8b011aa69854c8f14] to archive 2023-03-08 07:56:25,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:25,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/239ce4ae31ab466f825bc93bef3233cc to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/239ce4ae31ab466f825bc93bef3233cc 2023-03-08 07:56:25,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/9bb360071a134eb6827508736f7fed68 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/9bb360071a134eb6827508736f7fed68 2023-03-08 07:56:25,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8770ae2705de4cf7a06e987c7061f1ef to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/8770ae2705de4cf7a06e987c7061f1ef 2023-03-08 07:56:25,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/64f12bf27fe14747a5c7899553809725 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/64f12bf27fe14747a5c7899553809725 2023-03-08 07:56:25,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/2fa884e6ce7341a8b011aa69854c8f14 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/2fa884e6ce7341a8b011aa69854c8f14 2023-03-08 07:56:25,590 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308248935f766264f5aa75f1e588102b81b_del 2023-03-08 07:56:25,590 INFO [Listener at localhost/42999] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308248935f766264f5aa75f1e588102b81b_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308248935f766264f5aa75f1e588102b81b_del 2023-03-08 07:56:26,003 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc7b7796880b454a9cbd5c4bfebf9282 2023-03-08 07:56:26,004 DEBUG [Listener at localhost/42999] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/.tmp/family2/bc7b7796880b454a9cbd5c4bfebf9282 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/bc7b7796880b454a9cbd5c4bfebf9282 2023-03-08 07:56:26,008 INFO [Listener at localhost/42999] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc7b7796880b454a9cbd5c4bfebf9282 2023-03-08 07:56:26,009 INFO [Listener at localhost/42999] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 19bdaae25e528e021ec383cd49c4c05d/family2 of 19bdaae25e528e021ec383cd49c4c05d into bc7b7796880b454a9cbd5c4bfebf9282(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 07:56:26,009 DEBUG [Listener at localhost/42999] regionserver.HRegion(2289): Compaction status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:26,012 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:56:26,014 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:56:26,018 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:56:26,019 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:56:26,023 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35671] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x80\xCE\x89\xF9\x7FW\xEAZ\xBF\x96V\xD3\xB2\xDF\x82\x12"\x10} &\xEB%#\x9B\xCC\xCB\x80\xE8F\x11\xA0\x1F\xD6*\x10S$\xDE\x08\xA7\xEBn\xEF\xD6\xAE\xA1=\x15|\x1CK2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-08 07:56:26,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/3a9ffeba2ceb46978b7a26dd171fe4fa, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/734ed4a6524f4493a599586e1201ead1, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/8caa8d930bed4340bdb43999f7c531df, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/e26925ce394340ff9d3526de68f84025, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/57bf2b67d4f945eb979ae59ffc41e9da] to archive 2023-03-08 07:56:26,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:26,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/3a9ffeba2ceb46978b7a26dd171fe4fa to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/3a9ffeba2ceb46978b7a26dd171fe4fa 2023-03-08 07:56:26,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/734ed4a6524f4493a599586e1201ead1 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/734ed4a6524f4493a599586e1201ead1 2023-03-08 07:56:26,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/8caa8d930bed4340bdb43999f7c531df to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/8caa8d930bed4340bdb43999f7c531df 2023-03-08 07:56:26,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/e26925ce394340ff9d3526de68f84025 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/e26925ce394340ff9d3526de68f84025 2023-03-08 07:56:26,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/57bf2b67d4f945eb979ae59ffc41e9da to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family2/57bf2b67d4f945eb979ae59ffc41e9da 2023-03-08 07:56:26,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 07:56:26,045 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 07:56:26,045 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:56:26,045 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678262786045 2023-03-08 07:56:26,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 07:56:26,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-08 07:56:26,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del 2023-03-08 07:56:26,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del 2023-03-08 07:56:26,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del 2023-03-08 07:56:26,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:26,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] zookeeper.ReadOnlyZKClient(139): Connect 0x69b82ad6 to 127.0.0.1:52387 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 07:56:26,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51971a7b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 07:56:26,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:27,073 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303082288f7a177fa4fb59a08a3b7bd2ae82f to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303082288f7a177fa4fb59a08a3b7bd2ae82f 2023-03-08 07:56:27,074 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:27,085 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 07:56:27,088 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41822, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 07:56:27,097 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 07:56:27,099 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56802, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 07:56:27,139 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/83744b8958c34fd3b753c936a76ef158 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 07:56:27,139 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/83744b8958c34fd3b753c936a76ef158}] 2023-03-08 07:56:27,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/83744b8958c34fd3b753c936a76ef158 for inclusion in 86e34d59ea54b118fb72d7e3271d969b/family1 2023-03-08 07:56:27,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 07:56:27,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 07:56:27,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:27,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/83744b8958c34fd3b753c936a76ef158 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__luhco2nk1akqtga1ld2h4s4t85o418ne5ijmtf6fvl312clirev93g67lqlj6svu/family1/83744b8958c34fd3b753c936a76ef158 2023-03-08 07:56:27,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__luhco2nk1akqtga1ld2h4s4t85o418ne5ijmtf6fvl312clirev93g67lqlj6svu/family1/83744b8958c34fd3b753c936a76ef158 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/33bce98c72654dbe9b2d89d72d821bbf_SeqId_28_ 2023-03-08 07:56:27,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__luhco2nk1akqtga1ld2h4s4t85o418ne5ijmtf6fvl312clirev93g67lqlj6svu/family1/83744b8958c34fd3b753c936a76ef158 into 86e34d59ea54b118fb72d7e3271d969b/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/33bce98c72654dbe9b2d89d72d821bbf_SeqId_28_ - updating store file list. 2023-03-08 07:56:27,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/33bce98c72654dbe9b2d89d72d821bbf_SeqId_28_ into 86e34d59ea54b118fb72d7e3271d969b/family1 2023-03-08 07:56:27,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__luhco2nk1akqtga1ld2h4s4t85o418ne5ijmtf6fvl312clirev93g67lqlj6svu/family1/83744b8958c34fd3b753c936a76ef158 into 86e34d59ea54b118fb72d7e3271d969b/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/family1/33bce98c72654dbe9b2d89d72d821bbf_SeqId_28_) 2023-03-08 07:56:27,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__luhco2nk1akqtga1ld2h4s4t85o418ne5ijmtf6fvl312clirev93g67lqlj6svu/family1/83744b8958c34fd3b753c936a76ef158 2023-03-08 07:56:27,198 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:27,199 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:27,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303083eb4674dd8ae44be977226ddd961d854 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303083eb4674dd8ae44be977226ddd961d854 2023-03-08 07:56:27,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308724da24191a44c15b02c2f81ed6f4478 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308724da24191a44c15b02c2f81ed6f4478 2023-03-08 07:56:27,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088b505ea389b04b3e9946262470654add to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088b505ea389b04b3e9946262470654add 2023-03-08 07:56:27,202 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030899150c80546a4441a13758a5d458914e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030899150c80546a4441a13758a5d458914e 2023-03-08 07:56:27,202 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 07:56:27,202 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 07:56:27,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:27,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:27,744 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b6d43082b2de4a38be9c1f8ec030c588 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308b6d43082b2de4a38be9c1f8ec030c588 2023-03-08 07:56:27,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:27,761 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 07:56:27,785 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6826db040201426fbdba18cc93fb4adf first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 07:56:27,786 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6826db040201426fbdba18cc93fb4adf}] 2023-03-08 07:56:27,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6826db040201426fbdba18cc93fb4adf for inclusion in d5966061f30112fde4757a0c8d47e8af/family1 2023-03-08 07:56:27,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 07:56:27,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 07:56:27,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:27,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6826db040201426fbdba18cc93fb4adf to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__8ftqc8vesmj0sp997tf3c2hofvunavpfiteejtb4imtuokgmdpk376bevqgbioer/family1/6826db040201426fbdba18cc93fb4adf 2023-03-08 07:56:27,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__8ftqc8vesmj0sp997tf3c2hofvunavpfiteejtb4imtuokgmdpk376bevqgbioer/family1/6826db040201426fbdba18cc93fb4adf as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/c7566de41ce44a6c95411db32a751cb0_SeqId_28_ 2023-03-08 07:56:27,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__8ftqc8vesmj0sp997tf3c2hofvunavpfiteejtb4imtuokgmdpk376bevqgbioer/family1/6826db040201426fbdba18cc93fb4adf into d5966061f30112fde4757a0c8d47e8af/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/c7566de41ce44a6c95411db32a751cb0_SeqId_28_ - updating store file list. 2023-03-08 07:56:27,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/c7566de41ce44a6c95411db32a751cb0_SeqId_28_ into d5966061f30112fde4757a0c8d47e8af/family1 2023-03-08 07:56:27,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__8ftqc8vesmj0sp997tf3c2hofvunavpfiteejtb4imtuokgmdpk376bevqgbioer/family1/6826db040201426fbdba18cc93fb4adf into d5966061f30112fde4757a0c8d47e8af/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/family1/c7566de41ce44a6c95411db32a751cb0_SeqId_28_) 2023-03-08 07:56:27,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__8ftqc8vesmj0sp997tf3c2hofvunavpfiteejtb4imtuokgmdpk376bevqgbioer/family1/6826db040201426fbdba18cc93fb4adf 2023-03-08 07:56:27,834 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:27,834 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:27,836 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080c069cd712f840878342ef19226dbd82 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080c069cd712f840878342ef19226dbd82 2023-03-08 07:56:27,836 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d6f46f88334c66b3d3019e29ecbb1e to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d6f46f88334c66b3d3019e29ecbb1e 2023-03-08 07:56:27,837 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a451a2bb820946dabe588369e0b71bd6 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a451a2bb820946dabe588369e0b71bd6 2023-03-08 07:56:27,838 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308e0e105fe53f74905b66326acbc5ba99b to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308e0e105fe53f74905b66326acbc5ba99b 2023-03-08 07:56:27,838 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 07:56:27,838 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 07:56:27,838 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:27,838 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:28,782 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308bdf73e60c4a240bd88bf715c55c3370c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308bdf73e60c4a240bd88bf715c55c3370c 2023-03-08 07:56:28,783 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:28,832 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/849b3bb056ef4da7a8c6baba44d3c87c first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 07:56:28,832 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d., hostname=jenkins-hbase4.apache.org,33669,1678262016084, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/849b3bb056ef4da7a8c6baba44d3c87c}] 2023-03-08 07:56:28,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(763): Validating hfile at hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/849b3bb056ef4da7a8c6baba44d3c87c for inclusion in 19bdaae25e528e021ec383cd49c4c05d/family1 2023-03-08 07:56:28,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 07:56:28,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 07:56:28,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegion(2446): Flush status journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:28,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/849b3bb056ef4da7a8c6baba44d3c87c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__3tqnkua1dk5lbrj3po7jqpku04lpn2k786451soth2g71onfdieib55gpib5pdqr/family1/849b3bb056ef4da7a8c6baba44d3c87c 2023-03-08 07:56:28,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__3tqnkua1dk5lbrj3po7jqpku04lpn2k786451soth2g71onfdieib55gpib5pdqr/family1/849b3bb056ef4da7a8c6baba44d3c87c as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/389d609e1e224f20977279f0cdeb24a6_SeqId_28_ 2023-03-08 07:56:28,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(849): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__3tqnkua1dk5lbrj3po7jqpku04lpn2k786451soth2g71onfdieib55gpib5pdqr/family1/849b3bb056ef4da7a8c6baba44d3c87c into 19bdaae25e528e021ec383cd49c4c05d/family1 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/389d609e1e224f20977279f0cdeb24a6_SeqId_28_ - updating store file list. 2023-03-08 07:56:28,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(882): Loaded HFile hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/389d609e1e224f20977279f0cdeb24a6_SeqId_28_ into 19bdaae25e528e021ec383cd49c4c05d/family1 2023-03-08 07:56:28,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.HStore(855): Successfully loaded hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__3tqnkua1dk5lbrj3po7jqpku04lpn2k786451soth2g71onfdieib55gpib5pdqr/family1/849b3bb056ef4da7a8c6baba44d3c87c into 19bdaae25e528e021ec383cd49c4c05d/family1 (new location: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/family1/389d609e1e224f20977279f0cdeb24a6_SeqId_28_) 2023-03-08 07:56:28,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/staging/jenkins__testMajorCompactionFromAdmin__3tqnkua1dk5lbrj3po7jqpku04lpn2k786451soth2g71onfdieib55gpib5pdqr/family1/849b3bb056ef4da7a8c6baba44d3c87c 2023-03-08 07:56:28,881 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:28,881 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:28,882 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303081f5c1e263c124b71baae60a9447280b9 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303081f5c1e263c124b71baae60a9447280b9 2023-03-08 07:56:28,883 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085e761be0854345f887ce2f6e6343c52c to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085e761be0854345f887ce2f6e6343c52c 2023-03-08 07:56:28,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308aa57de8b7f264627981cd6bee9315ec3 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308aa57de8b7f264627981cd6bee9315ec3 2023-03-08 07:56:28,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308fdf0f4d66c8e4339a0e42d41916f4601 to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308fdf0f4d66c8e4339a0e42d41916f4601 2023-03-08 07:56:28,885 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 07:56:28,885 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 07:56:28,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:28,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 07:56:28,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 07:56:28,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del] 2023-03-08 07:56:28,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:28,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089c0166f3864e4d3c9381518d92ffcec4_del 2023-03-08 07:56:28,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 07:56:28,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del] 2023-03-08 07:56:28,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:28,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8538a3054bd4bc494bdbb85d947dd8a_del 2023-03-08 07:56:28,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 07:56:28,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del] 2023-03-08 07:56:28,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 07:56:28,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del to hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd84258310084b4e86598e9d9fc60d62_del 2023-03-08 07:56:28,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 07:56:28,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678262019891] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 07:56:28,890 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 07:56:28,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.8470 sec 2023-03-08 07:56:28,906 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:56:28,907 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:56:28,910 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 07:56:28,910 DEBUG [Listener at localhost/42999] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 07:56:28,915 INFO [Listener at localhost/42999] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 07:56:28,952 INFO [Listener at localhost/42999] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=248 (was 261), OpenFileDescriptor=655 (was 652) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=139 (was 161), ProcessCount=172 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=7464 (was 7426) - AvailableMemoryMB LEAK? - 2023-03-08 07:56:28,952 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x480b078d to 127.0.0.1:52387 2023-03-08 07:56:28,952 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:28,952 INFO [Listener at localhost/42999] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-08 07:56:28,953 INFO [Listener at localhost/42999] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:56:28,953 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57919e99 to 127.0.0.1:52387 2023-03-08 07:56:28,953 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:28,953 DEBUG [Listener at localhost/42999] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-08 07:56:28,953 DEBUG [Listener at localhost/42999] util.JVMClusterUtil(257): Found active master hash=551309655, stopped=false 2023-03-08 07:56:28,953 INFO [Listener at localhost/42999] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:56:28,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 07:56:28,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 07:56:28,955 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:56:28,955 INFO [Listener at localhost/42999] procedure2.ProcedureExecutor(629): Stopping 2023-03-08 07:56:28,956 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 07:56:28,956 DEBUG [Listener at localhost/42999] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3aefaa6e to 127.0.0.1:52387 2023-03-08 07:56:28,956 DEBUG [Listener at localhost/42999] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:28,956 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 07:56:28,956 INFO [Listener at localhost/42999] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33669,1678262016084' ***** 2023-03-08 07:56:28,956 INFO [Listener at localhost/42999] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-08 07:56:28,957 INFO [RS:0;jenkins-hbase4:33669] regionserver.HeapMemoryManager(220): Stopping 2023-03-08 07:56:28,957 INFO [RS:0;jenkins-hbase4:33669] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-08 07:56:28,957 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-08 07:56:28,957 INFO [RS:0;jenkins-hbase4:33669] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-08 07:56:28,957 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 3cf54bc577fa89b1f72695aed8375944 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for a495beea80e2b5d5dc82eeff331974f0 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for eba46d84043812856e232d435ccca635 2023-03-08 07:56:28,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3cf54bc577fa89b1f72695aed8375944, disabling compactions & flushes 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 38efdc462c389fa50e647861e0aace4c 2023-03-08 07:56:28,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for d5966061f30112fde4757a0c8d47e8af 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for b63cf1f82efef5926c4509ac90187cb1 2023-03-08 07:56:28,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for e906f28e8a2d2e81f61b506d78f3c6d3 2023-03-08 07:56:28,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. after waiting 0 ms 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 05308de2ff490d772d5a663e6692c790 2023-03-08 07:56:28,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 86e34d59ea54b118fb72d7e3271d969b 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 88f4fcc1aaf1e4b2430d092d92ce36a8 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 2fb06f0faeeffea363aea80614ef1a51 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 19bdaae25e528e021ec383cd49c4c05d 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 9b26a0fbb0144612b31a0a0058f9f1bf 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for f3f6a8571aa8d2b03ba727aae5b43603 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for a50cf6d148fcdc856ce86b2cdca8ceb8 2023-03-08 07:56:28,958 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 4a2a1f49d57eee4dc8b34f34c4c02a5c 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 8edf959f1d606a2cc1723c3487e05ae1 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for e9fb34d1dad0ba5071bdda7f618f683e 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 806d93567d24c9f9a1e9b8e642f88903 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:28,959 DEBUG [RS:0;jenkins-hbase4:33669] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x749237b6 to 127.0.0.1:52387 2023-03-08 07:56:28,959 DEBUG [RS:0;jenkins-hbase4:33669] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-08 07:56:28,959 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-08 07:56:28,960 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-08 07:56:28,960 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 07:56:28,960 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 07:56:28,960 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 07:56:28,960 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 07:56:28,960 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 07:56:28,960 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-08 07:56:28,960 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1505): Waiting on 05308de2ff490d772d5a663e6692c790, 1588230740, 19bdaae25e528e021ec383cd49c4c05d, 2fb06f0faeeffea363aea80614ef1a51, 38efdc462c389fa50e647861e0aace4c, 3cf54bc577fa89b1f72695aed8375944, 4a2a1f49d57eee4dc8b34f34c4c02a5c, 806d93567d24c9f9a1e9b8e642f88903, 86e34d59ea54b118fb72d7e3271d969b, 88f4fcc1aaf1e4b2430d092d92ce36a8, 8edf959f1d606a2cc1723c3487e05ae1, 9b26a0fbb0144612b31a0a0058f9f1bf, a495beea80e2b5d5dc82eeff331974f0, a50cf6d148fcdc856ce86b2cdca8ceb8, b63cf1f82efef5926c4509ac90187cb1, d5966061f30112fde4757a0c8d47e8af, e906f28e8a2d2e81f61b506d78f3c6d3, e9fb34d1dad0ba5071bdda7f618f683e, eba46d84043812856e232d435ccca635, f3f6a8571aa8d2b03ba727aae5b43603, fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:56:28,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/3cf54bc577fa89b1f72695aed8375944/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 07:56:28,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:56:28,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3cf54bc577fa89b1f72695aed8375944: 2023-03-08 07:56:28,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678262060775.3cf54bc577fa89b1f72695aed8375944. 2023-03-08 07:56:28,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a495beea80e2b5d5dc82eeff331974f0, disabling compactions & flushes 2023-03-08 07:56:28,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:56:28,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:56:28,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. after waiting 0 ms 2023-03-08 07:56:28,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:56:28,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithWeeklyPolicy/a495beea80e2b5d5dc82eeff331974f0/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 07:56:28,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:56:28,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a495beea80e2b5d5dc82eeff331974f0: 2023-03-08 07:56:28,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678262048173.a495beea80e2b5d5dc82eeff331974f0. 2023-03-08 07:56:28,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eba46d84043812856e232d435ccca635, disabling compactions & flushes 2023-03-08 07:56:28,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:56:28,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:56:28,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. after waiting 0 ms 2023-03-08 07:56:28,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:56:28,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/eba46d84043812856e232d435ccca635/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 07:56:28,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:56:28,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eba46d84043812856e232d435ccca635: 2023-03-08 07:56:28,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678262140820.eba46d84043812856e232d435ccca635. 2023-03-08 07:56:28,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 38efdc462c389fa50e647861e0aace4c, disabling compactions & flushes 2023-03-08 07:56:28,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:56:28,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:56:28,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. after waiting 0 ms 2023-03-08 07:56:28,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:56:28,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/38efdc462c389fa50e647861e0aace4c/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 07:56:28,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:56:28,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 38efdc462c389fa50e647861e0aace4c: 2023-03-08 07:56:28,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678262020036.38efdc462c389fa50e647861e0aace4c. 2023-03-08 07:56:28,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d5966061f30112fde4757a0c8d47e8af, disabling compactions & flushes 2023-03-08 07:56:28,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:28,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:28,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. after waiting 0 ms 2023-03-08 07:56:28,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:28,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/d5966061f30112fde4757a0c8d47e8af/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 07:56:28,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:28,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d5966061f30112fde4757a0c8d47e8af: 2023-03-08 07:56:28,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678262171260.d5966061f30112fde4757a0c8d47e8af. 2023-03-08 07:56:28,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b63cf1f82efef5926c4509ac90187cb1, disabling compactions & flushes 2023-03-08 07:56:28,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:56:28,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:56:28,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. after waiting 0 ms 2023-03-08 07:56:28,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:56:28,984 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-08 07:56:28,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b63cf1f82efef5926c4509ac90187cb1/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 07:56:28,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:56:28,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b63cf1f82efef5926c4509ac90187cb1: 2023-03-08 07:56:28,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678262094550.b63cf1f82efef5926c4509ac90187cb1. 2023-03-08 07:56:28,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e906f28e8a2d2e81f61b506d78f3c6d3, disabling compactions & flushes 2023-03-08 07:56:28,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:56:28,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:56:28,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. after waiting 0 ms 2023-03-08 07:56:28,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:56:28,991 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyPolicy/e906f28e8a2d2e81f61b506d78f3c6d3/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-08 07:56:28,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:56:28,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e906f28e8a2d2e81f61b506d78f3c6d3: 2023-03-08 07:56:28,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678262065781.e906f28e8a2d2e81f61b506d78f3c6d3. 2023-03-08 07:56:28,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 05308de2ff490d772d5a663e6692c790, disabling compactions & flushes 2023-03-08 07:56:28,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:56:28,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:56:28,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. after waiting 0 ms 2023-03-08 07:56:28,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:56:28,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/05308de2ff490d772d5a663e6692c790/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 07:56:29,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:56:29,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 05308de2ff490d772d5a663e6692c790: 2023-03-08 07:56:29,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678262140820.05308de2ff490d772d5a663e6692c790. 2023-03-08 07:56:29,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 86e34d59ea54b118fb72d7e3271d969b, disabling compactions & flushes 2023-03-08 07:56:29,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:29,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:29,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. after waiting 0 ms 2023-03-08 07:56:29,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:29,003 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 07:56:29,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/86e34d59ea54b118fb72d7e3271d969b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 07:56:29,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 86e34d59ea54b118fb72d7e3271d969b: 2023-03-08 07:56:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678262171260.86e34d59ea54b118fb72d7e3271d969b. 2023-03-08 07:56:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 88f4fcc1aaf1e4b2430d092d92ce36a8, disabling compactions & flushes 2023-03-08 07:56:29,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:56:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:56:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. after waiting 0 ms 2023-03-08 07:56:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:56:29,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/88f4fcc1aaf1e4b2430d092d92ce36a8/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 07:56:29,011 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:56:29,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 88f4fcc1aaf1e4b2430d092d92ce36a8: 2023-03-08 07:56:29,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678262078281.88f4fcc1aaf1e4b2430d092d92ce36a8. 2023-03-08 07:56:29,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2fb06f0faeeffea363aea80614ef1a51, disabling compactions & flushes 2023-03-08 07:56:29,011 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:56:29,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:56:29,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. after waiting 0 ms 2023-03-08 07:56:29,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:56:29,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/2fb06f0faeeffea363aea80614ef1a51/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 07:56:29,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:56:29,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2fb06f0faeeffea363aea80614ef1a51: 2023-03-08 07:56:29,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678262114841.2fb06f0faeeffea363aea80614ef1a51. 2023-03-08 07:56:29,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 19bdaae25e528e021ec383cd49c4c05d, disabling compactions & flushes 2023-03-08 07:56:29,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:29,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:29,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. after waiting 0 ms 2023-03-08 07:56:29,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:29,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionFromAdmin/19bdaae25e528e021ec383cd49c4c05d/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 07:56:29,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:29,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 19bdaae25e528e021ec383cd49c4c05d: 2023-03-08 07:56:29,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678262171260.19bdaae25e528e021ec383cd49c4c05d. 2023-03-08 07:56:29,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9b26a0fbb0144612b31a0a0058f9f1bf, disabling compactions & flushes 2023-03-08 07:56:29,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:56:29,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:56:29,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. after waiting 0 ms 2023-03-08 07:56:29,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:56:29,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 9b26a0fbb0144612b31a0a0058f9f1bf 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-08 07:56:29,036 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-08 07:56:29,161 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1505): Waiting on 1588230740, 4a2a1f49d57eee4dc8b34f34c4c02a5c, 806d93567d24c9f9a1e9b8e642f88903, 8edf959f1d606a2cc1723c3487e05ae1, 9b26a0fbb0144612b31a0a0058f9f1bf, a50cf6d148fcdc856ce86b2cdca8ceb8, e9fb34d1dad0ba5071bdda7f618f683e, f3f6a8571aa8d2b03ba727aae5b43603, fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:56:29,361 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1505): Waiting on 1588230740, 4a2a1f49d57eee4dc8b34f34c4c02a5c, 806d93567d24c9f9a1e9b8e642f88903, 8edf959f1d606a2cc1723c3487e05ae1, 9b26a0fbb0144612b31a0a0058f9f1bf, a50cf6d148fcdc856ce86b2cdca8ceb8, e9fb34d1dad0ba5071bdda7f618f683e, f3f6a8571aa8d2b03ba727aae5b43603, fbe4dd1fee45d9edc4c4fe869fc49ac1 2023-03-08 07:56:29,370 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/.tmp/info/25d6090a4e424952b7e5dc66a61d3440 2023-03-08 07:56:29,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25d6090a4e424952b7e5dc66a61d3440 2023-03-08 07:56:29,425 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/.tmp/info/de8f254d654f4838ae362d19065035b9 2023-03-08 07:56:29,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/.tmp/info/de8f254d654f4838ae362d19065035b9 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/info/de8f254d654f4838ae362d19065035b9 2023-03-08 07:56:29,435 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/info/de8f254d654f4838ae362d19065035b9, entries=3, sequenceid=7, filesize=4.8 K 2023-03-08 07:56:29,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 9b26a0fbb0144612b31a0a0058f9f1bf in 417ms, sequenceid=7, compaction requested=false 2023-03-08 07:56:29,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/namespace/9b26a0fbb0144612b31a0a0058f9f1bf/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-08 07:56:29,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:56:29,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9b26a0fbb0144612b31a0a0058f9f1bf: 2023-03-08 07:56:29,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678262019161.9b26a0fbb0144612b31a0a0058f9f1bf. 2023-03-08 07:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f3f6a8571aa8d2b03ba727aae5b43603, disabling compactions & flushes 2023-03-08 07:56:29,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. after waiting 0 ms 2023-03-08 07:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:56:29,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/ns/testMinorCompaction/f3f6a8571aa8d2b03ba727aae5b43603/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 07:56:29,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:56:29,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f3f6a8571aa8d2b03ba727aae5b43603: 2023-03-08 07:56:29,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678262140820.f3f6a8571aa8d2b03ba727aae5b43603. 2023-03-08 07:56:29,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a50cf6d148fcdc856ce86b2cdca8ceb8, disabling compactions & flushes 2023-03-08 07:56:29,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:56:29,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:56:29,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. after waiting 0 ms 2023-03-08 07:56:29,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/a50cf6d148fcdc856ce86b2cdca8ceb8/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 07:56:29,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a50cf6d148fcdc856ce86b2cdca8ceb8: 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678262114841.a50cf6d148fcdc856ce86b2cdca8ceb8. 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4a2a1f49d57eee4dc8b34f34c4c02a5c, disabling compactions & flushes 2023-03-08 07:56:29,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. after waiting 0 ms 2023-03-08 07:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/4a2a1f49d57eee4dc8b34f34c4c02a5c/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 07:56:29,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4a2a1f49d57eee4dc8b34f34c4c02a5c: 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678262060775.4a2a1f49d57eee4dc8b34f34c4c02a5c. 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fbe4dd1fee45d9edc4c4fe869fc49ac1, disabling compactions & flushes 2023-03-08 07:56:29,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. after waiting 0 ms 2023-03-08 07:56:29,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:56:29,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMajorCompactionWithMonthlyPolicy/fbe4dd1fee45d9edc4c4fe869fc49ac1/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 07:56:29,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:56:29,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fbe4dd1fee45d9edc4c4fe869fc49ac1: 2023-03-08 07:56:29,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678262034438.fbe4dd1fee45d9edc4c4fe869fc49ac1. 2023-03-08 07:56:29,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8edf959f1d606a2cc1723c3487e05ae1, disabling compactions & flushes 2023-03-08 07:56:29,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:29,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:29,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. after waiting 0 ms 2023-03-08 07:56:29,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:29,456 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testMinorCompactionWithMonthlyPolicy/8edf959f1d606a2cc1723c3487e05ae1/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 07:56:29,456 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:29,456 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8edf959f1d606a2cc1723c3487e05ae1: 2023-03-08 07:56:29,456 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678262158724.8edf959f1d606a2cc1723c3487e05ae1. 2023-03-08 07:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e9fb34d1dad0ba5071bdda7f618f683e, disabling compactions & flushes 2023-03-08 07:56:29,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. after waiting 0 ms 2023-03-08 07:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:56:29,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testCompactionWithHFileLink/e9fb34d1dad0ba5071bdda7f618f683e/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 07:56:29,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:56:29,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e9fb34d1dad0ba5071bdda7f618f683e: 2023-03-08 07:56:29,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678262114841.e9fb34d1dad0ba5071bdda7f618f683e. 2023-03-08 07:56:29,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 806d93567d24c9f9a1e9b8e642f88903, disabling compactions & flushes 2023-03-08 07:56:29,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:56:29,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:56:29,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. after waiting 0 ms 2023-03-08 07:56:29,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:56:29,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/default/testScannerOnBulkLoadRefHFiles/806d93567d24c9f9a1e9b8e642f88903/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 07:56:29,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:56:29,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 806d93567d24c9f9a1e9b8e642f88903: 2023-03-08 07:56:29,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678262060775.806d93567d24c9f9a1e9b8e642f88903. 2023-03-08 07:56:29,561 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 07:56:29,761 DEBUG [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 07:56:29,780 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/.tmp/rep_barrier/c948ccc1419747c881496dbbee1d9002 2023-03-08 07:56:29,783 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c948ccc1419747c881496dbbee1d9002 2023-03-08 07:56:29,789 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/.tmp/table/930e6f869c44415e8248d26a6008d098 2023-03-08 07:56:29,792 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 930e6f869c44415e8248d26a6008d098 2023-03-08 07:56:29,792 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/.tmp/info/25d6090a4e424952b7e5dc66a61d3440 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/info/25d6090a4e424952b7e5dc66a61d3440 2023-03-08 07:56:29,796 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25d6090a4e424952b7e5dc66a61d3440 2023-03-08 07:56:29,796 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/info/25d6090a4e424952b7e5dc66a61d3440, entries=226, sequenceid=130, filesize=37.9 K 2023-03-08 07:56:29,797 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/.tmp/rep_barrier/c948ccc1419747c881496dbbee1d9002 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/rep_barrier/c948ccc1419747c881496dbbee1d9002 2023-03-08 07:56:29,802 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c948ccc1419747c881496dbbee1d9002 2023-03-08 07:56:29,802 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/rep_barrier/c948ccc1419747c881496dbbee1d9002, entries=1, sequenceid=130, filesize=5.0 K 2023-03-08 07:56:29,803 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/.tmp/table/930e6f869c44415e8248d26a6008d098 as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/table/930e6f869c44415e8248d26a6008d098 2023-03-08 07:56:29,806 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 930e6f869c44415e8248d26a6008d098 2023-03-08 07:56:29,806 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/table/930e6f869c44415e8248d26a6008d098, entries=27, sequenceid=130, filesize=6.6 K 2023-03-08 07:56:29,809 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 849ms, sequenceid=130, compaction requested=false 2023-03-08 07:56:29,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-08 07:56:29,814 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 07:56:29,814 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 07:56:29,814 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 07:56:29,814 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-08 07:56:29,836 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-08 07:56:29,836 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-08 07:56:29,962 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33669,1678262016084; all regions closed. 2023-03-08 07:56:29,968 DEBUG [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/oldWALs 2023-03-08 07:56:29,968 INFO [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33669%2C1678262016084.meta:.meta(num 1678262018904) 2023-03-08 07:56:29,971 DEBUG [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/oldWALs 2023-03-08 07:56:29,971 INFO [RS:0;jenkins-hbase4:33669] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33669%2C1678262016084:(num 1678262017983) 2023-03-08 07:56:29,972 DEBUG [RS:0;jenkins-hbase4:33669] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:29,972 INFO [RS:0;jenkins-hbase4:33669] regionserver.LeaseManager(133): Closed leases 2023-03-08 07:56:29,972 INFO [RS:0;jenkins-hbase4:33669] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-08 07:56:29,972 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 07:56:29,973 INFO [RS:0;jenkins-hbase4:33669] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33669 2023-03-08 07:56:29,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33669,1678262016084 2023-03-08 07:56:29,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 07:56:29,980 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 07:56:29,980 ERROR [Listener at localhost/42999-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@141cf079 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 07:56:29,981 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33669,1678262016084] 2023-03-08 07:56:29,982 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33669,1678262016084; numProcessing=1 2023-03-08 07:56:29,983 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33669,1678262016084 already deleted, retry=false 2023-03-08 07:56:29,983 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33669,1678262016084 expired; onlineServers=0 2023-03-08 07:56:29,983 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35671,1678262014900' ***** 2023-03-08 07:56:29,983 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-08 07:56:29,984 DEBUG [M:0;jenkins-hbase4:35671] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61e15b6c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-08 07:56:29,984 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35671,1678262014900 2023-03-08 07:56:29,984 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35671,1678262014900; all regions closed. 2023-03-08 07:56:29,984 DEBUG [M:0;jenkins-hbase4:35671] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:29,984 DEBUG [M:0;jenkins-hbase4:35671] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-08 07:56:29,984 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-08 07:56:29,984 DEBUG [M:0;jenkins-hbase4:35671] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-08 07:56:29,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678262017639] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678262017639,5,FailOnTimeoutGroup] 2023-03-08 07:56:29,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678262017640] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678262017640,5,FailOnTimeoutGroup] 2023-03-08 07:56:29,984 INFO [M:0;jenkins-hbase4:35671] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-08 07:56:29,984 INFO [M:0;jenkins-hbase4:35671] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-08 07:56:29,984 INFO [M:0;jenkins-hbase4:35671] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-08 07:56:29,985 DEBUG [M:0;jenkins-hbase4:35671] master.HMaster(1512): Stopping service threads 2023-03-08 07:56:29,985 INFO [M:0;jenkins-hbase4:35671] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-08 07:56:29,985 ERROR [M:0;jenkins-hbase4:35671] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-03-08 07:56:29,985 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-08 07:56:29,985 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 07:56:29,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 07:56:29,986 INFO [M:0;jenkins-hbase4:35671] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-08 07:56:29,986 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-08 07:56:29,986 DEBUG [M:0;jenkins-hbase4:35671] zookeeper.ZKUtil(398): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-08 07:56:29,986 WARN [M:0;jenkins-hbase4:35671] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-08 07:56:29,986 INFO [M:0;jenkins-hbase4:35671] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-08 07:56:29,987 INFO [M:0;jenkins-hbase4:35671] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-08 07:56:29,987 DEBUG [M:0;jenkins-hbase4:35671] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 07:56:29,987 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:56:29,987 DEBUG [M:0;jenkins-hbase4:35671] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:56:29,987 DEBUG [M:0;jenkins-hbase4:35671] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 07:56:29,987 DEBUG [M:0;jenkins-hbase4:35671] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:56:29,987 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.50 KB heapSize=598.22 KB 2023-03-08 07:56:30,082 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 07:56:30,082 INFO [RS:0;jenkins-hbase4:33669] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33669,1678262016084; zookeeper connection closed. 2023-03-08 07:56:30,082 ERROR [Listener at localhost/42999-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@17524c39 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 07:56:30,082 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): regionserver:33669-0x100c31dfa1f0001, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 07:56:30,082 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@42f1be53] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@42f1be53 2023-03-08 07:56:30,082 ERROR [Listener at localhost/42999-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6fa70151 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 07:56:30,082 INFO [Listener at localhost/42999] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-08 07:56:30,394 INFO [M:0;jenkins-hbase4:35671] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.50 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a89803a581844dc9bdaf8fce060c7bea 2023-03-08 07:56:30,398 INFO [M:0;jenkins-hbase4:35671] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a89803a581844dc9bdaf8fce060c7bea 2023-03-08 07:56:30,398 DEBUG [M:0;jenkins-hbase4:35671] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a89803a581844dc9bdaf8fce060c7bea as hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a89803a581844dc9bdaf8fce060c7bea 2023-03-08 07:56:30,401 INFO [M:0;jenkins-hbase4:35671] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a89803a581844dc9bdaf8fce060c7bea 2023-03-08 07:56:30,401 INFO [M:0;jenkins-hbase4:35671] regionserver.HStore(1080): Added hdfs://localhost:39077/user/jenkins/test-data/bed4a46f-45b9-e904-9588-6c18857e811b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a89803a581844dc9bdaf8fce060c7bea, entries=123, sequenceid=1005, filesize=33.3 K 2023-03-08 07:56:30,402 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegion(2948): Finished flush of dataSize ~512.50 KB/524802, heapSize ~598.20 KB/612560, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1005, compaction requested=false 2023-03-08 07:56:30,402 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 07:56:30,402 DEBUG [M:0;jenkins-hbase4:35671] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 07:56:30,405 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 07:56:30,405 INFO [M:0;jenkins-hbase4:35671] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-08 07:56:30,406 INFO [M:0;jenkins-hbase4:35671] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35671 2023-03-08 07:56:30,408 DEBUG [M:0;jenkins-hbase4:35671] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,35671,1678262014900 already deleted, retry=false 2023-03-08 07:56:30,509 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 07:56:30,509 INFO [M:0;jenkins-hbase4:35671] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35671,1678262014900; zookeeper connection closed. 2023-03-08 07:56:30,509 ERROR [Listener at localhost/42999-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@245988a3 rejected from java.util.concurrent.ThreadPoolExecutor@57efd19e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 07:56:30,509 DEBUG [Listener at localhost/42999-EventThread] zookeeper.ZKWatcher(600): master:35671-0x100c31dfa1f0000, quorum=127.0.0.1:52387, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 07:56:30,510 ERROR [Listener at localhost/42999-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@d55da24 rejected from java.util.concurrent.ThreadPoolExecutor@57efd19e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 07:56:30,511 WARN [Listener at localhost/42999] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-08 07:56:30,514 INFO [Listener at localhost/42999] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 07:56:30,622 WARN [BP-476193878-172.31.14.131-1678262011522 heartbeating to localhost/127.0.0.1:39077] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-08 07:56:30,622 WARN [BP-476193878-172.31.14.131-1678262011522 heartbeating to localhost/127.0.0.1:39077] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-476193878-172.31.14.131-1678262011522 (Datanode Uuid 45ea35ad-08cb-4f1a-a4db-6690f1026251) service to localhost/127.0.0.1:39077 2023-03-08 07:56:30,624 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/cluster_884a5b1f-9982-fdc0-2356-490fbc1a75f2/dfs/data/data1/current/BP-476193878-172.31.14.131-1678262011522] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 07:56:30,625 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/40e069e4-0871-3847-f46e-087109334491/cluster_884a5b1f-9982-fdc0-2356-490fbc1a75f2/dfs/data/data2/current/BP-476193878-172.31.14.131-1678262011522] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 07:56:30,641 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 07:56:30,641 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69b82ad6 to 127.0.0.1:52387 2023-03-08 07:56:30,641 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 07:56:30,662 INFO [Listener at localhost/42999] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 07:56:30,792 INFO [Listener at localhost/42999] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-08 07:56:30,889 INFO [Listener at localhost/42999] hbase.HBaseTestingUtility(1293): Minicluster is down